Optical skin detection for face unlock

ABSTRACT

A method for face authentication is proposed. The method includes at least one face detection step including determining at least one first image by using at least one camera, determining at least one material property from a second image, where the second image is recorded while projecting at least one illumination pattern including a plurality of illumination features on the scene, and at least one authentication step including authenticating the detected face by using the face detection and the material property.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of U.S. application Ser.No. 18/264,326, filed Aug. 4, 2023, which is a U.S. National PhaseApplication of International Patent Application No. PCT/EP22/053941,filed Feb. 17, 2022, which claims priority to EP Patent Application No.21157800.0, filed Feb. 18, 2021, each of which is hereby incorporated byreference herein.

DESCRIPTION Field of the Invention

The invention relates to a method for face authentication, a mobiledevice and various uses of the method. The devices, methods and usesaccording to the present invention specifically may be employed forexample in various areas of daily life, security technology, gaming,traffic technology, production technology, photography such as digitalphotography or video photography for arts, documentation or technicalpurposes, safety technology, information technology, agriculture, cropprotection, maintenance, cosmetics, medical technology or in thesciences. However, other applications are also possible.

Prior Art

In today's digital world secure access to information technology is anessential requirement of any state-of-the-art system. Standard conceptslike passphrases or PIN codes are currently extended, combined or evenreplaced by biometric methods like fingerprint or face recognition.Although passphrases can provide a high level of security if chosencarefully and at adequate length, this step requires attention and theneed of memorizing several potentially long phrases depending on the ITlandscape. Furthermore, it is never guaranteed that the person providingthe passphrase is authorized by the owner of the passphrase or thedigital device. In contrast biometric features like faces orfingerprints are unique and person-specific properties. Therefore, usingfeatures derived from these not only can be more convenient thanpassphrases/PIN codes but also more secure since they combine personalidentity with an unlock process.

Unfortunately, similarly to being able to spy passwords, fingerprintsand faces can be also artificially created in order to impersonate alegitimate user. Automatic face recognition tools of the firstgeneration use digital camera images or image streams and apply 2D imageprocessing methods to extract characteristic features, while machinelearning techniques are used to generate face templates to performidentity recognition based on those features. The second generation offace recognition algorithms uses deep convolutional neuronal networksinstead of hand-crafted image features to generate the classificationmodel.

However, both approaches can be attacked, e.g., with a high-qualityphotograph that is nowadays freely available to be downloaded from theInternet. Therefore, the idea of presentation attack detection (PAD)gained significance. Early approaches were designed to protect againstsimple attacks like presenting the photograph of a legitimate user byrecording a sequence of images and test for time dependent features, forexample small but natural changes in the head position or eye blinking.These methods can again be tricked by playing a pre-recorded video ofthe user or by careful animations generated from publicly availablephotographs. In order to exclude displays as potential spoof objects,near infra-red (NIR) cameras can be used, since displays emit photonsonly in the visible regime of the electromagnetic spectrum. As a furthercounteraction, 3D cameras were introduced, which can clearly distinguishbetween planar photos or tablets with playback videos and a 3D face.Still, these systems can be attacked with high quality masks generated,for example, by 3D printing, careful 3D-arrangement of 2D photos, orhand-crafted silicone or latex masks to name a few approaches. Sincemasks are typically worn by humans, liveness detection based on smallmotions fails as well. However, these types of masks can be rejected bya PAD system that is able to classify human skin from other materialssuch as described in EP application No. 20159984.2 filed on Feb. 28,2020 and EP application No. 20 154 961.5190679 filed on Jan. 31, 2020,the full content of which is included by reference.

A further problem might arise from differences in skin opticalproperties with respect ethnic origin. A reliable recognition and PADtechnology needs to be fully agnostic with respect to these differentorigins.

Besides security considerations, also the speed of the unlock processand the required computational power are important to provide anacceptable user experience. High-speed face recognition can be used forseveral tasks after the successful unlock of a device, for example tocheck whether the user is still in front of the display or to launchfurther secure applications like banking apps by performing an on thefly check of the person in front of the display. Again, speed andcomputational resources have great impact on the user experience.

Current 3D algorithms are computationally very demanding andpresentation attack detections require several video frames to beprocessed. Therefore, expensive hardware needs to be involved to deliveracceptable unlock performance. Furthermore, high power consumption is aconsequence.

Summarizing, current methods for face unlock do not provide the abilityto reliably detect spoof attacks by 3D masks and perform this task at aspeed below human detection limit.

US 2019/213309 A1 describes a system and method of authenticating auser's face with a ranging sensor. The ranging sensor includes a time offlight sensor and a reflectance sensor. The ranging sensor transmits asignal that is reflected off of a user and received back at the rangingsensor. The received signal can be used to determine distance betweenthe user and the sensor, and the reflectance value of the user. With thedistance or the reflectivity, a processor can activate a facialrecognition process in response to the distance and the reflectivity.

Problem Addressed by the Invention

It is therefore an object of the present invention to provide devicesand methods facing the above-mentioned technical challenges of knowndevices and methods. Although simple presentation attacks withphotographs and videos of the legitimate face can be detected, anapproach to reliably detect presentation attacks with 3D face masks isstill missing. Specifically, another layer of security is needed toenable the replacement of passphrases/PIN codes by biometric featuresderived from faces to unlock digital devices. Moreover, a method isneeded that operates fully agnostic with respect to different skin typesoriginating from different ethnic origins.

SUMMARY OF THE INVENTION

This problem is solved by the invention with the features of theindependent patent claims. Advantageous developments of the invention,which can be realized individually or in combination, are presented inthe dependent claims and/or in the following specification and detailedembodiments.

As used in the following, the terms “have”, “comprise” or “include” orany arbitrary grammatical variations thereof are used in a non-exclusiveway. Thus, these terms may both refer to a situation in which, besidesthe feature introduced by these terms, no further features are presentin the entity described in this context and to a situation in which oneor more further features are present. As an example, the expressions “Ahas B”, “A comprises B” and “A includes B” may both refer to a situationin which, besides B, no other element is present in A (i.e. a situationin which A solely and exclusively consists of B) and to a situation inwhich, besides B, one or more further elements are present in entity A,such as element C, elements C and D or even further elements.

Further, it shall be noted that the terms “at least one”, “one or more”or similar expressions indicating that a feature or element may bepresent once or more than once typically will be used only once whenintroducing the respective feature or element. In the following, in mostcases, when referring to the respective feature or element, theexpressions “at least one” or “one or more” will not be repeated,non-withstanding the fact that the respective feature or element may bepresent once or more than once.

Further, as used in the following, the terms “preferably”, “morepreferably”, “particularly”, “more particularly”, “specifically”, “morespecifically” or similar terms are used in conjunction with optionalfeatures, without restricting alternative possibilities. Thus, featuresintroduced by these terms are optional features and are not intended torestrict the scope of the claims in any way. The invention may, as theskilled person will recognize, be performed by using alternativefeatures. Similarly, features introduced by “in an embodiment of theinvention”or similar expressions are intended to be optional features,without any restriction regarding alternative embodiments of theinvention, without any restrictions regarding the scope of the inventionand without any restriction regarding the possibility of combining thefeatures introduced in such a way with other optional or non-optionalfeatures of the invention.

In a first aspect of the present invention a method for faceauthentication is disclosed. The face to be authenticated may be,specifically, a human face. The term “face authentication” as usedherein is a broad term and is to be given its ordinary and customarymeaning to a person of ordinary skill in the art and is not to belimited to a special or customized meaning. The term specifically mayrefer, without limitation, to verifying a recognized object or part ofthe recognized object as human face. Specifically, the authenticationmay comprise distinguishing a real human face from attack materials thatwere produced to mimic a face. The authentication may comprise verifyingidentity of a respective user and/or assigning identity to a user. Theauthentication may comprise generating and/or providing identityinformation, e.g. to other devices such as to at least one authorizationdevice for authorization for access of a mobile device, a machine, anautomobile, a building or the like. The identify information may beproofed by the authentication. For example, the identity information maybe and/or may comprise at least one identity token. In case ofsuccessful authentication the recognized object or part of therecognized object is verified to be a real face and/or the identity ofthe object, in particular a user, is verified.

The method comprises the following steps:

-   -   a) at least one face detection step, wherein the face detection        step comprises determining at least one first image by using at        least one camera, wherein the first image comprises at least one        two-dimensional image of a scene suspected to comprise the face,        wherein the face detection step comprises detecting the face in        the first image by identifying in the first image at least one        pre-defined or pre-determined geometrical feature characteristic        for faces by using at least one processing unit;    -   b) at least one skin detection step, wherein the skin detection        step comprises projecting at least one illumination pattern        comprising a plurality of illumination features on the scene by        using at least one illumination unit and determining at least        one second image using the at least one camera, wherein the        second image comprises a plurality of reflection features        generated by the scene in response to illumination by the        illumination features, wherein each of the reflection features        comprises at least one beam profile, wherein the skin detection        step comprises determining a first beam profile information of        at least one of the reflection features located inside an image        region of the second image corresponding to an image region of        the first image comprising the identified geometrical feature by        analysis of its beam profile and determining at least one        material property of the reflection feature from the first beam        profile information by using the processing unit, wherein the        detected face is characterized as skin if the material property        corresponds to at least one property characteristic for skin;    -   c) at least one 3D detection step, wherein the 3D detection step        comprises determining a second beam profile information of at        least four of the reflection features located inside the image        region of the second image corresponding to the image region of        the first image comprising the identified geometrical feature by        analysis of their beam profiles and determining at least one        depth level from the second beam profile information of said        reflection features by using the processing unit, wherein the        detected face is characterized as 3D object if the depth level        deviates from a pre-determined or pre-defined depth level of        plane objects;    -   d) at least one authentication step, wherein the authentication        step comprises authenticating the detected face by using at        least one authentication unit if in step b) the detected face is        characterized as skin and in step c) the detected face is        characterized as 3D object.

The method steps may be performed in the given order or may be performedin a different order. Further, one or more additional method steps maybe present which are not listed. Further, one, more than one or even allof the method steps may be performed repeatedly.

The term “camera” as used herein is a broad term and is to be given itsordinary and customary meaning to a person of ordinary skill in the artand is not to be limited to a special or customized meaning. The termspecifically may refer, without limitation, to a device having at leastone imaging element configured for recording or capturing spatiallyresolved onedimensional, two-dimensional or even three-dimensionaloptical data or information. The camera may be a digital camera. As anexample, the camera may comprise at least one camera chip, such as atleast one CCD chip and/or at least one CMOS chip configured forrecording images. The camera may be or may comprise at least one nearinfrared camera.

As used herein, without limitation, the term “image” specifically mayrelate to data recorded by using a camera, such as a plurality ofelectronic readings from the imaging device, such as the pixels of thecamera chip. The camera, besides the at least one camera chip or imagingchip, may comprise further elements, such as one or more opticalelements, e.g. one or more lenses. As an example, the camera may be afix-focus camera, having at least one lens which is fixedly adjustedwith respect to the camera. Alternatively, however, the camera may alsocomprise one or more variable lenses which may be adjusted,automatically or manually.

The camera may be a camera of a mobile device such as of notebookcomputers, tablets or, specifically, cell phones such as smart phonesand the like. Thus, specifically, the camera may be part of a mobiledevice which, besides the at least one camera, comprises one or moredata processing devices such as one or more data processors. Othercameras, however, are feasible. The term “mobile device” as used hereinis a broad term and is to be given its ordinary and customary meaning toa person of ordinary skill in the art and is not to be limited to aspecial or customized meaning. The term specifically may refer, withoutlimitation, to a mobile electronics device, more specifically to amobile communication device such as a cell phone or smart phone.Additionally or alternatively, the mobile device may also refer to atablet computer or another type of portable computer.

Specifically, the camera may be or may comprise at least one opticalsensor having at least one light-sensitive area. As used herein, an“optical sensor” generally refers to a light-sensitive device fordetecting a light beam, such as for detecting an illumination and/or alight spot generated by at least one light beam. As further used herein,a “light-sensitive area” generally refers to an area of the opticalsensor which may be illuminated externally, by the at least one lightbeam, in response to which illumination at least one sensor signal isgenerated. The light-sensitive area may specifically be located on asurface of the respective optical sensor. Other embodiments, however,are feasible. The camera may comprise a plurality of optical sensorseach having a light sensitive area. As used herein, the term “theoptical sensors each having at least one light sensitive area” refers toconfigurations with a plurality of single optical sensors each havingone light sensitive area and to configurations with one combined opticalsensor having a plurality of light sensitive areas. The term “opticalsensor” furthermore refers to a light-sensitive device configured togenerate one output signal. In case the camera comprises a plurality ofoptical sensors, each optical sensor may be embodied such that preciselyone light-sensitive area is present in the respective optical sensor,such as by providing precisely one light-sensitive area which may beilluminated, in response to which illumination precisely one uniformsensor signal is created for the whole optical sensor. Thus, eachoptical sensor may be a single area optical sensor. The use of thesingle area optical sensors, however, renders the setup of the cameraspecifically simple and efficient. Thus, as an example, commerciallyavailable photo-sensors, such as commercially available siliconphotodiodes, each having precisely one sensitive area, may be used inthe set-up. Other embodiments, however, are feasible.

The optical sensor specifically may be or may comprise at least onephotodetector, preferably inorganic photodetectors, more preferablyinorganic semiconductor photodetectors, most preferably siliconphotodetectors. Specifically, the optical sensor may be sensitive in theinfrared spectral range. The optical sensor may comprise at least onesensor element comprising a matrix of pixels. All pixels of the matrixor at least a group of the optical sensors of the matrix specificallymay be identical. Groups of identical pixels of the matrix specificallymay be provided for different spectral ranges, or all pixels may beidentical in terms of spectral sensitivity. Further, the pixels may beidentical in size and/or with regard to their electronic oroptoelectronic properties. Specifically, the optical sensor may be ormay comprise at least one array of inorganic photodiodes which aresensitive in the infrared spectral range, preferably in the range of 700nm to 3.0 micrometers. Specifically, the optical sensor may be sensitivein the part of the near infrared region where silicon photodiodes areapplicable specifically in the range of 700 nm to 1100 nm. Infraredoptical sensors which may be used for optical sensors may becommercially available infrared optical sensors, such as infraredoptical sensors commercially available under the brand name Hertzstueck™from trinamiX™ GmbH, D-67056 Ludwigshafen am Rhein, Germany. Thus, as anexample, the optical sensor may comprise at least one optical sensor ofan intrinsic photovoltaic type, more preferably at least onesemiconductor photodiode selected from the group consisting of: a Gephotodiode, an InGaAs photodiode, an extended InGaAs photodiode, an InAsphotodiode, an InSb photodiode, a HgCdTe photodiode. Additionally oralternatively, the optical sensor may comprise at least one opticalsensor of an extrinsic photovoltaic type, more preferably at least onesemiconductor photodiode selected from the group consisting of: a Ge:Auphotodiode, a Ge:Hg photodiode, a Ge:Cu photodiode, a Ge:Zn photodiode,a Si:Ga photodiode, a Si:As photodiode. Additionally or alternatively,the optical sensor may comprise at least one photoconductive sensor suchas a PbS or PbSe sensor, a bolometer, preferably a bolometer selectedfrom the group consisting of a VO bolometer and an amorphous Sibolometer.

The optical sensor may be sensitive in one or more of the ultraviolet,the visible or the infrared spectral range. Specifically, the opticalsensor may be sensitive in the visible spectral range from 500 nm to 780nm, most preferably at 650 nm to 750 nm or at 690 nm to 700 nm.Specifically, the optical sensor may be sensitive in the near infraredregion. Specifically, the optical sensor may be sensitive in the part ofthe near infrared region where silicon photodiodes are applicablespecifically in the range of 700 nm to 1000 nm. The optical sensor,specifically, may be sensitive in the infrared spectral range,specifically in the range of 780 nm to 3.0 micrometers. For example, theoptical sensor each, independently, may be or may comprise at least oneelement selected from the group consisting of a photodiode, a photocell,a photoconductor, a phototransistor or any combination thereof. Forexample, the optical sensor may be or may comprise at least one elementselected from the group consisting of a CCD sensor element, a CMOSsensor element, a photodiode, a photocell, a photoconductor, aphototransistor or any combination thereof. Any other type ofphotosensitive element may be used. The photosensitive element generallymay fully or partially be made of inorganic materials and/or may fullyor partially be made of organic materials. Most commonly, one or morephotodiodes may be used, such as commercially available photodiodes,e.g. inorganic semiconductor photodiodes.

The optical sensor may comprise at least one sensor element comprising amatrix of pixels. Thus, as an example, the optical sensor may be part ofor constitute a pixelated optical device. For example, the opticalsensor may be and/or may comprise at least one CCD and/or CMOS device.As an example, the optical sensor may be part of or constitute at leastone CCD and/or CMOS device having a matrix of pixels, each pixel forminga light-sensitive area.

As used herein, the term “sensor element” generally refers to a deviceor a combination of a plurality of devices configured for sensing atleast one parameter. In the present case, the parameter specifically maybe an optical parameter, and the sensor element specifically may be anoptical sensor element. The sensor element may be formed as a unitary,single device or as a combination of several devices. The sensor elementcomprises a matrix of optical sensors. The sensor element may compriseat least one CMOS sensor. The matrix may be composed of independentpixels such as of independent optical sensors. Thus, a matrix ofinorganic photodiodes may be composed. Alternatively, however, acommercially available matrix may be used, such as one or more of a CCDdetector, such as a CCD detector chip, and/or a CMOS detector, such as aCMOS detector chip. Thus, generally, the sensor element may be and/ormay comprise at least one CCD and/or CMOS device and/or the opticalsensors may form a sensor array or may be part of a sensor array, suchas the above-mentioned matrix. Thus, as an example, the sensor elementmay comprise an array of pixels, such as a rectangular array, having mrows and n columns, with m, n, independently, being positive integers.Preferably, more than one column and more than one row is given, i.e.n>1, m>1. Thus, as an example, n may be 2 to 16 or higher and m may be 2to 16 or higher. Preferably, the ratio of the number of rows and thenumber of columns is close to 1. As an example, n and m may be selectedsuch that 0.3 s m/n s 3, such as by choosing m/n=1:1, 4:3, 16:9 orsimilar. As an example, the array may be a square array, having an equalnumber of rows and columns, such as by choosing m=2, n=2 or m=3, n=3 orthe like.

The matrix may be composed of independent pixels such as of independentoptical sensors. Thus, a matrix of inorganic photodiodes may becomposed. Alternatively, however, a commercially available matrix may beused, such as one or more of a CCD detector, such as a CCD detectorchip, and/or a CMOS detector, such as a CMOS detector chip. Thus,generally, the optical sensor may be and/or may comprise at least oneCCD and/or CMOS device and/or the optical sensors of the camera form asensor array or may be part of a sensor array, such as theabove-mentioned matrix.

The matrix specifically may be a rectangular matrix having at least onerow, preferably a plurality of rows, and a plurality of columns. As anexample, the rows and columns may be oriented essentially perpendicular.As used herein, the term “essentially perpendicular” refers to thecondition of a perpendicular orientation, with a tolerance of e.g. ±20°or less, preferably a tolerance of ±10° or less, more preferably atolerance of ±5° or less. Similarly, the term “essentially parallel”refers to the condition of a parallel orientation, with a tolerance ofe.g. ±20° or less, preferably a tolerance of ±10° or less, morepreferably a tolerance of ±5° or less. Thus, as an example, tolerancesof less than 20°, specifically less than 10° or even less than 5°, maybe acceptable. In order to provide a wide range of view, the matrixspecifically may have at least 10 rows, preferably at least 500 rows,more preferably at least 1000 rows. Similarly, the matrix may have atleast 10 columns, preferably at least 500 columns, more preferably atleast 1000 columns. The matrix may comprise at least 50 optical sensors,preferably at least 100000 optical sensors, more preferably at least5000000 optical sensors. The matrix may comprise a number of pixels in amulti-mega pixel range. Other embodiments, however, are feasible. Thus,in setups in which an axial rotational symmetry is to be expected,circular arrangements or concentric arrangements of the optical sensorsof the matrix, which may also be referred to as pixels, may bepreferred.

Thus, as an example, the sensor element may be part of or constitute apixelated optical device. For example, the sensor element may be and/ormay comprise at least one CCD and/or CMOS device. As an example, thesensor element may be part of or constitute at least one CCD and/or CMOSdevice having a matrix of pixels, each pixel forming a light-sensitivearea. The sensor element may employ a rolling shutter or global shuttermethod to read out the matrix of optical sensors.

The camera further may comprise at least one transfer device. The cameramay further comprise one or more additional elements such as one or moreadditional optical elements. The camera may comprise at least oneoptical element selected from the group consisting of: transfer device,such as at least one lens and/or at least one lens system, at least onediffractive optical element. The term “transfer device”, also denoted as“transfer system”, may generally refer to one or more optical elementswhich are adapted to modify the light beam, such as by modifying one ormore of a beam parameter of the light beam, a width of the light beam ora direction of the light beam. The transfer device may be adapted toguide the light beam onto the optical sensor. The transfer devicespecifically may comprise one or more of: at least one lens, for exampleat least one lens selected from the group consisting of at least onefocus-tunable lens, at least one aspheric lens, at least one sphericlens, at least one Fresnel lens; at least one diffractive opticalelement; at least one concave mirror; at least one beam deflectionelement, preferably at least one mirror; at least one beam splittingelement, preferably at least one of a beam splitting cube or a beamsplitting mirror; at least one multi-lens system. The transfer devicemay have a focal length. As used herein, the term “focal length” of thetransfer device refers to a distance over which incident collimated rayswhich may impinge the transfer device are brought into a “focus” whichmay also be denoted as “focal point”. Thus, the focal length constitutesa measure of an ability of the transfer device to converge an impinginglight beam. Thus, the transfer device may comprise one or more imagingelements which can have the effect of a converging lens. By way ofexample, the transfer device can have one or more lenses, in particularone or more refractive lenses, and/or one or more convex mirrors. Inthis example, the focal length may be defined as a distance from thecenter of the thin refractive lens to the principal focal points of thethin lens. For a converging thin refractive lens, such as a convex orbiconvex thin lens, the focal length may be considered as being positiveand may provide the distance at which a beam of collimated lightimpinging the thin lens as the transfer device may be focused into asingle spot. Additionally, the transfer device can comprise at least onewavelength-selective element, for example at least one optical filter.Additionally, the transfer device can be designed to impress apredefined beam profile on the electromagnetic radiation, for example,at the location of the sensor region and in particular the sensor area.The abovementioned optional embodiments of the transfer device can, inprinciple, be realized individually or in any desired combination.

The transfer device may have an optical axis. As used herein, the term“optical axis of the transfer device” generally refers to an axis ofmirror symmetry or rotational symmetry of the lens or lens system. Thetransfer system, as an example, may comprise at least one beam path,with the elements of the transfer system in the beam path being locatedin a rotationally symmetrical fashion with respect to the optical axis.Still, one or more optical elements located within the beam path mayalso be off-centered or tilted with respect to the optical axis. In thiscase, however, the optical axis may be defined sequentially, such as byinterconnecting the centers of the optical elements in the beam path,e.g. by interconnecting the centers of the lenses, wherein, in thiscontext, the optical sensors are not counted as optical elements. Theoptical axis generally may denote the beam path. Therein, the camera mayhave a single beam path along which a light beam may travel from theobject to the optical sensors, or may have a plurality of beam paths. Asan example, a single beam path may be given or the beam path may besplit into two or more partial beam paths. In the latter case, eachpartial beam path may have its own optical axis. In case of a pluralityof optical sensors, the optical sensors may be located in one and thesame beam path or partial beam path. Alternatively, however, the opticalsensors may also be located in different partial beam paths.

The transfer device may constitute a coordinate system, wherein alongitudinal coordinate is a coordinate along the optical axis andwherein d is a spatial offset from the optical axis. The coordinatesystem may be a polar coordinate system in which the optical axis of thetransfer device forms a z-axis and in which a distance from the z-axisand a polar angle may be used as additional coordinates. A directionparallel or antiparallel to the z-axis may be considered a longitudinaldirection, and a coordinate along the z-axis may be considered alongitudinal coordinate. Any direction perpendicular to the z-axis maybe considered a transversal direction, and the polar coordinate and/orthe polar angle may be considered a transversal coordinate.

The camera is configured for determining at least one image of thescene, in particular the first image. As used herein, the term “scene”may refer to a spatial region. The scene may comprise the face underauthentication and a surrounding environment. The first image itself maycomprise pixels, the pixels of the image correlating to pixels of thematrix of the sensor element. Consequently, when referring to “pixels”,reference is either made to the units of image information generated bythe single pixels of the sensor element or to the single pixels of thesensor element directly. The first image is at least one two-dimensionalimage. As used herein, the term “two dimensional image” may generallyrefer to an image having information about transversal coordinates suchas the dimensions of height and width. The first image may be an RGB(red green blue) image. The term “determining at least one first image”may refer to capturing and/or recording the first image.

The face detection step comprises detecting the face in the first imageby identifying in the first image the at least one pre-defined orpre-determined geometrical feature characteristic for faces by using theat least one processing unit. Specifically, the face detection stepcomprises detecting the face in the first image by identifying in thefirst image at least one pre-defined or pre-determined geometricalfeature which is characteristic for faces by using the at least oneprocessing unit.

As further used herein, the term “processing unit” generally refers toan arbitrary data processing device adapted to perform the namedoperations such as by using at least one processor and/or at least oneapplication-specific integrated circuit. Thus, as an example, the atleast one processing unit may comprise a software code stored thereoncomprising a number of computer commands. The processing unit mayprovide one or more hardware elements for performing one or more of thenamed operations and/or may provide one or more processors with softwarerunning thereon for performing one or more of the named operations.Operations, including evaluating the images may be performed by the atleast one processing unit. Thus, as an example, one or more instructionsmay be implemented in software and/or hardware. Thus, as an example, theprocessing unit may comprise one or more programmable devices such asone or more computers, application-specific integrated circuits (ASICs),Digital Signal Processors (DSPs), or Field Programmable Gate Arrays(FPGAs) which are configured to perform the above-mentioned evaluation.Additionally or alternatively, however, the processing unit may alsofully or partially be embodied by hardware. The processing unit and thecamera may fully or partially be integrated into a single device. Thus,generally, the processing unit also may form part of the camera.Alternatively, the processing unit and the camera may fully or partiallybe embodied as separate devices.

The processing unit may be or may comprise one or more integratedcircuits, such as one or more application-specific integrated circuits(ASICs), and/or one or more data processing devic-es, such as one ormore computers, preferably one or more microcomputers and/ormicrocontrollers, Field Programmable Arrays, or Digital SignalProcessors. Additional components may be comprised, such as one or morepreprocessing devices and/or data acquisition devices, such as one ormore devices for receiving and/or preprocessing of the sensor signals,such as one or more AD-converters and/or one or more filters. Further,the processing unit may comprise one or more measurement devices, suchas one or more measurement devices for measuring electrical currentsand/or electrical voltages. Further, the processing unit may compriseone or more data storage devices. Further, the processing unit maycomprise one or more interfaces, such as one or more wireless interfacesand/or one or more wire-bound interfaces.

The processing unit may be configured to one or more of displaying,visualizing, analyzing, distributing, communicating or furtherprocessing of information, such as information obtained by the camera.The processing unit, as an example, may be connected or incorporate atleast one of a display, a projector, a monitor, an LCD, a TFT, aloudspeaker, a multichannel sound system, an LED pattern, or a furthervisualization device. It may further be connected or incorporate atleast one of a communication device or communication interface, aconnector or a port, capable of sending encrypted or unencryptedinformation using one or more of email, text messages, telephone,Bluetooth, Wi-Fi, infrared or internet interfaces, ports or connections.It may further be connected to or incorporate at least one of aprocessor, a graphics processor, a CPU, an Open Multimedia ApplicationsPlatform (OMAP™), an integrated circuit, a system on a chip such asproducts from the Apple A series or the Samsung S3C2 series, amicrocontroller or microprocessor, one or more memory blocks such asROM, RAM, EEPROM, or flash memory, timing sources such as oscillators orphase-locked loops, counter-timers, real-time timers, or power-on resetgenerators, voltage regulators, power management circuits, or DMAcontrollers. Individual units may further be connected by buses such asAMBA buses or be integrated in an Internet of Things or Industry 4.0type network.

The processing unit may be connected by or have further externalinterfaces or ports such as one or more of serial or parallel interfacesor ports, USB, Centronics Port, FireWire, HDMI, Ethernet, Bluetooth,RFID, Wi-Fi, USART, or SPI, or analogue interfaces or ports such as oneor more of ADCs or DACs, or standardized interfaces or ports to furtherdevices such as a 2D-camera device using an RGB-interface such asCameraLink. The processing unit may further be connected by one or moreof interprocessor interfaces or ports, FPGA-FPGA-interfaces, or serialor parallel interfaces ports. The processing unit may further beconnected to one or more of an optical disc drive, a CD-RW drive, aDVD+RW drive, a flash drive, a memory card, a disk drive, a hard diskdrive, a solid state disk or a solid state hard disk.

The processing unit may be connected by or have one or more furtherexternal connectors such as one or more of phone connectors, RCAconnectors, VGA connectors, hermaphrodite connectors, USB connectors,HDMI connectors, 8P8C connectors, BCN connectors, IEC 60320 C14connectors, optical fiber connectors, D-subminiature connectors, RFconnectors, coaxial connectors, SCART connectors, XLR connectors, and/ormay incorporate at least one suitable socket for one or more of theseconnectors.

The detecting of the face in the first image may comprise identifyingthe at least one predefined or pre-determined geometrical featurecharacteristic for faces. The term “geometrical feature characteristicfor faces” as used herein is a broad term and is to be given itsordinary and customary meaning to a person of ordinary skill in the artand is not to be limited to a special or customized meaning. The termspecifically may refer, without limitation, to at least onegeometry-based feature which describe the shape of the face and itscomponents, in particular one or more of nose, eyes, mouth or eyebrowand the like. The processing unit may comprise at least one databasewherein the geometrical feature characteristic for faces are stored suchas in a lookup table. Techniques for identifying the at least onepre-defined or pre-determined geometrical feature characteristic forfaces are generally known to the skilled person. For example, the facedetection may be performed as described in Masi, Lacopo, et al. “Deepface recognition: A survey” 2018 31st SIBGRAPI conference on graphics,patterns and images (SIBGRAPI), IEEE, 2018, the full content of which isincluded by reference.

The processing unit may be configured for performing at least one imageanalysis and/or image processing in order to identify the geometricalfeature. The image analysis and/or image processing may use at least onefeature detection algorithm. The image analysis and/or image processingmay comprise one or more of the following: a filtering; a selection ofat least one region of interest; a background correction; adecomposition into color channels; a decomposition into hue, saturation,and/or brightness channels; a frequency decomposition; a singular valuedecomposition; applying a blob detector; applying a corner detector;applying a Determinant of Hessian filter; applying a principlecurvature-based region detector; applying a gradient location andorientation histogram algorithm; applying a histogram of orientedgradients descriptor; applying an edge detector; applying a differentialedge detector; applying a Canny edge detector; applying a Laplacian ofGaussian filter; applying a Difference of Gaussian filter; applying aSobel operator; applying a Laplace operator; applying a Scharr operator;applying a Prewitt operator; applying a Roberts operator; applying aKirsch operator; applying a high-pass filter; applying a low-passfilter; applying a Fourier transformation; applying aRadon-transformation; applying a Hough-transformation; applying awavelet-transformation; a thresholding; creating a binary image. Theregion of interest may be determined manually by a user or may bedetermined automatically, such as by recognizing a feature within thefirst image.

Specifically subsequent to the face detection step, the skin detectionstep may be performed comprising projecting at least one illuminationpattern comprising a plurality of illumination features on the scene byusing the at least one illumination unit. However, embodiments arefeasible wherein the skin detection step is performed before the facedetection step.

As used herein, the term “illumination unit”, also denoted asillumination source, may generally refers to at least one arbitrarydevice configured for generating at least one illumination pattern. Theillumination unit may be configured for providing the illuminationpattern for illumination of the scene. The illumination unit may beadapted to directly or indirectly illuminating the scene, wherein theillumination pattern is remitted, in particular reflected or scattered,by surfaces of the scene and, thereby, is at least partially directedtowards the camera. The illumination unit may be configured forilluminating the scene, for example, by directing a light beam towardsthe scene, which reflects the light beam. The illumination unit may beconfigured for generating an illuminating light beam for illuminatingthe scene.

The illumination unit may comprise at least one light source. Theillumination unit may comprise a plurality of light sources. Theillumination unit may comprise an artificial illumination source, inparticular at least one laser source and/or at least one incandescentlamp and/or at least one semiconductor light source, for example, atleast one light-emitting diode, in particular an organic and/orinorganic light-emitting diode. As an example, the light emitted by theillumination unit may have a wavelength of 300 to 1100 nm, especially500 to 1100 nm. Additionally or alternatively, light in the infraredspectral range may be used, such as in the range of 780 nm to 3.0 μm.Specifically, the light in the part of the near infrared region wheresilicon photodiodes are applicable specifically in the range of 700 nmto 1100 nm may be used.

The illumination unit may be configured for generating the at least oneillumination pattern in the infrared region. The illumination featuresmay have wavelengths in a near infrared (NIR) regime. The illuminationfeatures may have wavelengths of about 940 nm. At this wavelengthMelanin absorption runs out so that dark and light complecion reflectlight almost identical. However, other wavelength in the NIR region maybe possible such as one or more of 805 nm, 830 nm, 835 nm, 850 nm, 905nm, or 980 nm. Moreover, using light in the near infrared region allowsthat light is not or only weakly detected by human eyes and is stilldetectable by silicon sensors, in particular standard silicon sensors.

The illumination unit may be configured for emitting light at a singlewavelength. In other embodiments, the illumination unit may beconfigured for emitting light with a plurality of wavelengths allowingadditional measurements in other wavelengths channels.

As used herein, the term “ray” generally refers to a line that isperpendicular to wavefronts of light which points in a direction ofenergy flow. As used herein, the term “beam” generally refers to acollection of rays. In the following, the terms “ray” and “beam” will beused as synonyms. As further used herein, the term “light beam”generally refers to an amount of light, specifically an amount of lighttraveling essentially in the same direction, including the possibilityof the light beam having a spreading angle or widening angle. The lightbeam may have a spatial extension. Specifically, the light beam may havea non-Gaussian beam profile. The beam profile may be selected from thegroup consisting of a trapezoid beam profile; a triangle beam profile; aconical beam profile. The trapezoid beam profile may have a plateauregion and at least one edge region. The light beam specifically may bea Gaussian light beam or a linear combination of Gaussian light beams,as will be outlined in further detail below. Other embodiments arefeasible, however.

The illumination unit may be or may comprise at least one multiple beamlight source. For example, the illumination unit may comprise at leastone laser source and one or more diffractive optical elements (DOEs).Specifically, the illumination unit may comprise at least one laserand/or laser source. Various types of lasers may be employed, such assemiconductor lasers, double heterostructure lasers, external cavitylasers, separate confinement heterostructure lasers, quantum cascadelasers, distributed bragg reflector lasers, polariton lasers, hybridsilicon lasers, extended cavity diode lasers, quantum dot lasers, volumeBragg grating lasers, Indium Arsenide lasers, transistor lasers, diodepumped lasers, distributed feedback lasers, quantum well lasers,interband cascade lasers, Gallium Arsenide lasers, semiconductor ringlaser, extended cavity diode lasers, or vertical cavity surface-emittinglasers. Additionally or alternatively, non-laser light sources may beused, such as LEDs and/or light bulbs. The illumination unit maycomprise one or more diffractive optical elements (DOEs) adapted togenerate the illumination pattern. For example, the illumination unitmay be adapted to generate and/or to project a cloud of points, forexample the illumination unit may comprise one or more of at least onedigital light processing projector, at least one LCoS projector, atleast one spatial light modulator; at least one diffractive opticalelement; at least one array of light emitting diodes; at least one arrayof laser light sources. On account of their generally defined beamprofiles and other properties of handleability, the use of at least onelaser source as the illumination unit is particularly preferred. Theillumination unit may be integrated into a housing of the camera or maybe separated from the camera.

Further, the illumination unit may be configured for emitting modulatedor non-modulated light. In case a plurality of illumination units isused, the different illumination units may have different modulationfrequencies which later on may be used for distinguishing the lightbeams.

The light beam or light beams generated by the illumination unitgenerally may propagate parallel to the optical axis or tilted withrespect to the optical axis, e.g. including an angle with the opticalaxis. The illumination unit may be configured such that the light beamor light beams propagates from the illumination unit towards the scenealong an optical axis of the illumination unit and/or of the camera. Forthis purpose, the illumination unit and/or the camera may comprise atleast one reflective element, preferably at least one prism, fordeflecting the illuminating light beam onto the optical axis. As anexample, the light beam or light beams, such as the laser light beam,and the optical axis may include an angle of less than 10°, preferablyless than 5° or even less than 2°. Other embodiments, however, arefeasible. Further, the light beam or light beams may be on the opticalaxis or off the optical axis. As an example, the light beam or lightbeams may be parallel to the optical axis having a distance of less 10than 10 mm to the optical axis, preferably less than 5 mm to the opticalaxis or even less than 1 mm to the optical axis or may even coincidewith the optical axis.

As used herein, the term “at least one illumination pattern” refers toat least one arbitrary pattern comprising at least one illuminationfeature adapted to illuminate at least one part of the scene. As usedherein, the term “illumination feature” refers to at least one at leastpartially extended feature of the pattern. The illumination pattern maycomprise a single illumination feature. The illumination pattern maycomprise a plurality of illumination features. The illumination patternmay be selected from the group consisting of: at least one pointpattern; at least one line pattern; at least one stripe pattern; atleast one checkerboard pattern; at least one pattern comprising anarrangement of periodic or non periodic features. The illuminationpattern may comprise regular and/or constant and/or periodic patternsuch as a triangular pattern, a rectangular pattern, a hexagonal patternor a pattern comprising further convex tilings. The illumination patternmay exhibit the at least one illumination feature selected from thegroup consisting of: at least one point; at least one line; at least twolines such as parallel or crossing lines; at least one point and oneline; at least one arrangement of periodic or non-periodic feature; atleast one arbitrary shaped featured. The illumination pattern maycomprise at least one pattern selected from the group consisting of: atleast one point pattern, in particular a pseudo-random point pattern; arandom point pattern or a quasi random pattern; at least one Sobolpattern; at least one quasiperiodic pattern; at least one patterncomprising at least one pre-known feature at least one regular pattern;at least one triangular pattern; at least one hexagonal pattern; atleast one rectangular pattern at least one pattern comprising convexuniform tilings; at least one line pattern comprising at least one line;at least one line pattern comprising at least two lines such as parallelor crossing lines. For example, the illumination unit may be adapted togenerate and/or to project a cloud of points. The illumination unit maycomprise the at least one light projector adapted to generate a cloud ofpoints such that the illumination pattern may comprise a plurality ofpoint pattern. The illumination pattern may comprise a periodic grid oflaser spots. The illumination unit may comprise at least one maskadapted to generate the illumination pattern from at least one lightbeam generated by the illumination unit.

A distance between two features of the illumination pattern and/or anarea of the at least one illumination feature may depend on the circleof confusion in the image. As outlined above, the illumination unit maycomprise the at least one light source configured for generating the atleast one illumination pattern. Specifically, the illumination unitcomprises at least one laser source and/or at least one laser diodewhich is designated for generating laser radiation. The illuminationunit may comprise the at least one diffractive optical element (DOE).The illumination unit may comprise at least one point projector, such asthe at least one laser source and the DOE, adapted to project at leastone periodic point pattern. As further used herein, the term “projectingat least one illumination pattern” may refer to providing the at leastone illumination pattern for illuminating the at least one scene.

The skin detection step comprises determining the at least one secondimage, also denoted as reflection image, using the camera. The methodmay comprise determining plurality of second images. The reflectionfeatures of the plurality of second images may be used for skindetection in step b) and/or for 3D detection in step c).

The second image comprises a plurality of reflection features generatedby the scene in response to illumination by the illumination features.As used herein, the term “reflection feature” may refer to a feature inan image plane generated by the scene in response to illumination,specifically with at least one illumination feature. Each of thereflection features comprises at least one beam profile, also denotedreflection beam profile. As used herein, the term “beam profile” of thereflection feature may generally refer to at least one intensitydistribution of the reflection feature, such as of a light spot on theoptical sensor, as a function of the pixel. The beam profile may beselected from the group consisting of a trapezoid beam profile; atriangle beam profile; a conical beam profile and a linear combinationof Gaussian beam profiles.

The evaluation of the second image may comprise identifying thereflection features of the second image. The processing unit may beconfigured for performing at least one image analysis and/or imageprocessing in order to identify the reflection features. The imageanalysis and/or image processing may use at least one feature detectionalgorithm. The image analysis and/or image processing may comprise oneor more of the following: a filtering; a selection of at least oneregion of interest; a formation of a difference image between an imagecreated by the sensor signals and at least one offset; an inversion ofsensor signals by inverting an image created by the sensor signals; aformation of a difference image between an image created by the sensorsignals at different times; a background correction; a decompositioninto color channels; a decomposition into hue; saturation; andbrightness channels; a frequency decomposition; a singular valuedecomposition; applying a blob detector; applying a corner detector;applying a Determinant of Hessian filter; applying a principlecurvature-based region detector; applying a maximally stable extremalregions detector; applying a generalized Hough-transformation; applyinga ridge detector; applying an affine invariant feature detector;applying an affine-adapted interest point operator; applying a Harrisaffine region detector; applying a Hessian affine region detector;applying a scaleinvariant feature transform; applying a scale-spaceextrema detector; applying a local feature detector; applying speeded uprobust features algorithm; applying a gradient location and orientationhistogram algorithm; applying a histogram of oriented gradientsdescriptor; applying a Deriche edge detector; applying a differentialedge detector; applying a spatiotemporal interest point detector;applying a Moravec corner detector; applying a Canny edge detector;applying a Laplacian of Gaussian filter; applying a Difference ofGaussian filter; applying a Sobel operator; applying a Laplace operator;applying a Scharr operator; applying a Prewitt operator; applying aRoberts operator; applying a Kirsch operator; applying a high-passfilter; applying a low-pass filter; applying a Fourier transformation;applying a Radon-transformation; applying a Hough-transformation;applying a wavelet-transformation; a thresholding; creating a binaryimage. The region of interest may be determined manually by a user ormay be determined automatically, such as by recognizing a feature withinthe image generated by the optical sensor.

For example, the illumination unit may be configured for generatingand/or projecting a cloud of points such that a plurality of illuminatedregions is generated on the optical sensor, for example the CMOSdetector. Additionally, disturbances may be present on the opticalsensor such as disturbances due to speckles and/or extraneous lightand/or multiple reflections. The processing unit may be adapted todetermine at least one region of interest, for example one or morepixels illuminated by the light beam which are used for determination ofthe longitudinal coordinate for the respective reflection feature, whichwill be described in more detail below. For example, the processing unitmay be adapted to perform a filtering method, for example, ablob-analysis and/or an edge filter and/or object recognition method.

The processing unit may be configured for performing at least one imagecorrection. The image correction may comprise at least one backgroundsubtraction. The processing unit may be adapted to remove influencesfrom background light from the beam profile, for example, by an imagingwithout further illumination.

The processing unit may be configured for determining the beam profileof the respective reflection feature. As used herein, the term“determining the beam profile” refers to identifying at least onereflection feature provided by the optical sensor and/or selecting atleast one reflection feature provided by the optical sensor andevaluating at least one intensity distribution of the reflectionfeature. As an example, a region of the matrix may be used and evaluatedfor determining the intensity distribution, such as a three-dimensionalintensity distribution or a two-dimensional intensity distribution, suchas along an axis or line through the matrix. As an example, a center ofillumination by the light beam may be determined, such as by determiningthe at least one pixel having the highest illumination, and acrosssectional axis may be chosen through the center of illumination.The intensity distribution may an intensity distribution as a functionof a coordinate along this cross-sectional axis through the center ofillumination. Other evaluation algorithms are feasible.

The reflection feature may cover or may extend over at least one pixelof the second image. For example, the reflection feature may cover ormay extend over plurality of pixels. The processing unit may beconfigured for determining and/or for selecting all pixels connected toand/or belonging to the reflection feature, e.g. a light spot. Theprocessing unit may be configured for determining the center ofintensity by

${R_{coi} = \frac{1}{l \cdot {\sum{j \cdot r_{pixel}}}}},$

wherein R_(coi) is a position of center of intensity, r_(pixel) is thepixel position and l=Σ_(j)I_(total) with j being the number of pixels jconnected to and/or belonging to the reflection feature and I_(total)being the total intensity.

The processing unit is configured for determining a first beam profileinformation of at least one of the reflection features located inside animage region of the second image corresponding to an image region of thefirst image comprising the identified geometrical feature by analysis ofits beam profile. The method may comprise identifying the image regionof the second image corresponding to the image region of the first imagecomprising the identified geometrical feature. Specifically, the methodmay comprise matching pixels of the first image and the second image andselecting the pixels of the second image corresponding to the imageregion of the first image comprising the identified geometrical feature.The method may comprise considering in addition further reflectionfeatures located outside said image region of the second image.

As used herein, the term “beam profile information” may refer toarbitrary information and/or property derived from and/or relating tothe beam profile of the reflection feature. The first and the secondbeam profile information may be identical or may be different. Forexample, the first beam profile information may be an intensitydistribution, a reflection profile, a center of intensity, a materialfeature. For skin detection in step b), beam profile analysis may beused. Specifically, beam profile analysis makes use of reflectionproperties of coherent light projected onto object surfaces to classifymaterials. The classification of materials may be performed as describedin WO 2020/187719, in EP application 20159984.2 filed on Feb. 28, 2020and/or EP application 20 154 961.5 filed on Jan. 31, 2020, the fullcontent of which is included by reference. Specifically, a periodic gridof laser spots, e.g. a hexagonal grid as described in EP application 20170 905.2 filed on Apr. 22, 2020, is projected and the reflection imageis recorded with the camera. Analyzing the beam profile of eachreflection feature recorded by the camera may be performed byfeature-based methods. The feature-based methods may be explained in thefollowing. The feature based methods may be used in combination withmachine learning methods which may allow parametrization of a skinclassification model. Alternatively or in combination, convolutionalneuronal networks may be utilized to classify skin by using thereflection images as an input.

Other methods for authenticating a user's face are known, such as fromUS 2019/213309 A1. However, these methods use time of flight (ToF)sensors. A well-known working principle of a ToF sensor is sending outlight and measuring the time span until receiving the reflected light.In contrast, the proposed beam profile analysis uses a projectedillumination pattern. For ToF sensor using such a projected pattern isnot possible. Using illumination pattern may be advantageous e.g. inview of covering and therefore allowing to take into account differentlocations on the face. This may enhance reliability and security of theauthentication of the user's face.

The skin detection step may comprise determining at least one materialproperty of the reflection feature from the beam profile information byusing the processing unit. Specifically, the processing unit isconfigured for identifying a reflection feature as to be generated byilluminating biological tissue, in particular human skin, in case itsreflection beam profile fulfills at least one predetermined orpredefined criterion. As used herein, the term “at least onepredetermined or predefined criterion” refers to at least one propertyand/or value suitable to distinguish biological tissue, in particularhuman skin, from other materials. The predetermined or predefinedcriterion may be or may comprise at least one predetermined orpredefined value and/or threshold and/or threshold range referring to amaterial property.

The reflection feature may be indicated as to be generated by biologicaltissue in case the reflection beam profile fulfills the at least onepredetermined or predefined criterion. As used herein, the term“indicate” refers to an arbitrary indication such as an electronicsignal and/or at least one visual or acoustic indication. The processingunit is configured for identifying the reflection feature as to benon-skin otherwise. As used herein, the term “biological tissue”generally refers to biological material comprising living cells.Specifically, the processing unit may be configured for skin detection.The term “identification” of being generated by biological tissue, inparticular human skin, may refer to determining and/or validatingwhether a surface to be examined or under test is or comprisesbiological tissue, in particular human skin, and/or to distinguishbiological tissue, in particular human skin, from other tissues, inparticular other surfaces. The method according to the present inventionmay allow for distinguishing human skin from one or more of inorganictissue, metal surfaces, plastics surfaces, foam, paper, wood, a display,a screen, cloth. The method according to the present invention may allowfor distinguishing human biological tissue from surfaces of artificialor non-living objects.

The processing unit may be configured for determining the materialproperty m of the surface remitting the reflection feature by evaluatingthe beam profile of the reflection feature. As used herein, the term“material property” refers to at least one arbitrary property of thematerial configured for characterizing and/or identification and/orclassification of the material. For example, the material property maybe a property selected from the group consisting of: roughness,penetration depth of light into the material, a property characterizingthe material as biological or non-biological material, a reflectivity, aspecular reflectivity, a diffuse reflectivity, a surface property, ameasure for translucence, a scattering, specifically a back-scatteringbehavior or the like. The at least one material property may be aproperty selected from the group consisting of: a scatteringcoefficient, a translucency, a transparency, a deviation from aLambertian surface reflection, a speckle, and the like. As used herein,the term “determining at least one material property” may refer toassigning the material property to respective reflection feature, inparticular to the detected face. The processing unit may comprise atleast one database comprising a list and/or table, such as a lookup listor a lookup table, of predefined and/or predetermined materialproperties. The list and/or table of material properties may bedetermined and/or generated by performing at least one test measurement,for example by performing material tests using samples having knownmaterial properties. The list and/or table of material properties may bedetermined and/or generated at the manufacturer site and/or by a user.The material property may additionally be assigned to a materialclassifier such as one or more of a material name, a material group suchas biological or non-biological material, translucent or nontranslucentmaterials, metal or non-metal, skin or non-skin, fur or non-fur, carpetor noncarpet, reflective or non-reflective, specular reflective ornon-specular reflective, foam or non-foam, hair or non-hair, roughnessgroups or the like. The processing unit may comprise at least onedatabase comprising a list and/or table comprising the materialproperties and associated material name and/or material group.

The reflection properties of skin may be characterized by thesimultaneous occurrence of direct reflection at the surface(Lambertian-like) and subsurface scattering (volume scattering). Thisleads to a broadening of the laser spot on skin compared to theabove-mentioned materials.

The first beam profile information may be a reflection profile. Forexample, without wishing to be bound by this theory, human skin may havea reflection profile, also denoted back scattering profile, comprisingparts generated by back reflection of the surface, denoted as surfacereflection, and parts generated by very diffuse reflection from lightpenetrating the skin, denoted as diffuse part of the back reflection.With respect to reflection profile of human skin reference is made to“Lasertechnik in der Medizin: Grundlagen, Systeme, Anwendungen”,“Wirkung von Laserstrahlung auf Gewebe”, 1991, pages 10 171 to 266,Jürgen Eichler, Theo Seiler, Springer Verlag, ISBN 0939-0979. Thesurface reflection of the skin may increase with the wavelengthincreasing towards the near infrared. Further, the penetration depth mayincrease with increasing wavelength from visible to near infrared. Thediffuse part of the back reflection may increase with penetrating depthof the light. These properties may be used to distinguish skin fromother materials, by analyzing the back scattering profile.

Specifically, the processing unit may be configured for comparing thereflection beam profile with at least one predetermined and/orprerecorded and/or predefined beam profile. The predetermined and/orprerecorded and/or predefined beam profile may be stored in a table or alookup table and may be determined e.g. empirically, and may, as anexample, be stored in at least one data storage device of the detector.For example, the predetermined and/or prerecorded and/or predefined beamprofile may be determined during initial startup of a device executingthe method according to the present invention. For example, thepredetermined and/or prerecorded and/or predefined beam profile may bestored in at least one data storage device of the processing unit or thedevice, e.g. by software, specifically by the app downloaded from an appstore or the like. The reflection feature may be identified as to begenerated by biological tissue in case the reflection beam profile andthe predetermined and/or prerecorded and/or predefined beam profile areidentical. The comparison may comprise overlaying the reflection beamprofile and the predetermined or predefined beam profile such that theircenters of intensity match. The comparison may comprise determining adeviation, e.g. a sum of squared point to point distances, between thereflection beam profile and the predetermined and/or prerecorded and/orpredefined beam profile. The processing unit may be adapted to comparethe determined deviation with at least one threshold, wherein in casethe determined deviation is below and/or equal the threshold the surfaceis indicated as biological tissue and/or the detection of biologicaltissue is confirmed. The threshold value may be stored in a table or alookup table and may be determined e.g. empirically and may, as anexample, be stored in at least one data storage device of the processingunit.

Additionally or alternatively, the first beam profile information may bedetermined by applying at least one image filter to the image of thearea. As further used herein, the term “image” refers to atwo-dimensional function, f(x,y), wherein brightness and/or color valuesare given for any x,y-position in the image. The position may bediscretized corresponding to the recording pixels. The brightness and/orcolor may be discretized corresponding to a bit-depth of the opticalsensors. As used herein, the term “image filter” refers to at least onemathematical operation applied to the beam profile and/or to the atleast one specific region of the beam profile. Specifically, the imagefilter ϕ maps an image f, or a region of interest in the image, onto areal number, ϕ(f(x,y))=φ, wherein φ denotes a feature, in particular amaterial feature. Images may be subject to noise and the same holds truefor features. Therefore, features may be random variables. The featuresmay be normally distributed. If features are not normally distributed,they may be transformed to be normally distributed such as by aBox-Cox-Transformation.

The processing unit may be configured for determining at least onematerial feature ϕ_(2m) by applying at least one material dependentimage filter ϕ₂ to the image. As used herein, the term “materialdependent” image filter refers to an image having a material dependentoutput. The output of the material dependent image filter is denotedherein “material feature φ_(2m)” or “material dependent feature φ_(2m)”.The material feature may be or may comprise at least one informationabout the at least one material property of the surface of the scenehaving generated the reflection feature.

The material dependent image filter may be at least one filter selectedfrom the group consisting of: a luminance filter; a spot shape filter; asquared norm gradient; a standard deviation; a smoothness filter such asa Gaussian filter or median filter; a grey-level-occurrence-basedcontrast filter; a grey-level-occurrence-based energy filter; agrey-level-occurrence-based homogeneity filter; agrey-level-occurrence-based dissimilarity filter; a Law's energy filter;a threshold area filter; or a linear combination thereof; or a furthermaterial dependent image filter ϕ_(2other) which correlates to one ormore of the luminance filter, the spot shape filter, the squared normgradient, the standard deviation, the smoothness filter, thegrey-level-occurrence-based energy filter, thegrey-level-occurrence-based homogeneity filter, thegrey-level-occurrence-based dissimilarity filter, the Law's energyfilter, or the threshold area filter, or a linear combination thereof by|ρ_(ϕ2other,ϕm)|≥0.40 with ϕ_(m) being one of the luminance filter, thespot shape filter, the squared norm gradient, the standard deviation,the smoothness filter, the grey-level-occurrence-based energy filter,the grey-level-occurrence-based homogeneity filter, thegrey-level-occurrence-based dissimilarity filter, the Law's energyfilter, or the threshold area filter, or a linear combination thereof.The further material dependent image filter ϕ_(2other) may correlate toone or more of the material dependent image filters ϕ_(m) by|ρ_(ϕ2other,ϕm)|≥0.60, preferably by |ρ_(ϕ2other,ϕm)|≥0.80.

The material dependent image filter may be at least one arbitrary filterϕ that passes a hypothesis testing. As used herein, the term “passes ahypothesis testing” refers to the fact that a Null-hypothesis H₀ isrejected and an alternative hypothesis H₁ is accepted. The hypothesistesting may comprise testing the material dependency of the image filterby applying the image filter to a predefined data set. The data set maycomprise a plurality of beam profile images. As used herein, the term“beam profile image” refers to a sum of N_(B) Gaussian radial basisfunctions,

f _(k)(x,y)=|Σ_(l=0) ^(N) ^(B) ⁻¹ g _(lk)(x,y)|,

g _(lk)(x,y)=a _(lk) e ^(−(α(x-x) ^(lk) ⁾⁾ ² e ^(−(αy-y) ^(lk) ⁾⁾ ²

wherein each of the N_(B) Gaussian radial basis functions is defined bya center (x_(lk), y_(lk)), a prefactor, a_(lk), and an exponentialfactor α=1/∈. The exponential factor is identical for all Gaussianfunctions in all images. The center-positions, x_(lk), y_(lk), areidentical for all images f_(k): (x₀, x₁, . . . , x_(N) _(B-1) ), (y₀,y₁, . . . y_(N) _(B-1) ). Each of the beam profile images in the datasetmay correspond to a material classifier and a distance. The materialclassifier may be a label such as ‘Material A’, ‘Material B’, etc. Thebeam profile images may be generated by using the above formula forf_(k)(x,y) in combination with the following parameter table:

Image Material classifier, Index Material Index Distance z Parameters k= 0 Skin, m = 0 0.4 m (a₀₀, a₁₀, . . . , a_(N) _(B) ⁻¹⁰ ₎ k = 1 Skin, m= 0 0.6 m (a₀₁, a₁₁, . . . , a_(N) _(B) ⁻¹¹ ₎ k = 2 Fabric, m = 1 0.6 m(a₀₂, a₁₂, . . . , a_(N) _(B) ⁻¹² ₎ . . . . . . k = N Material J, m = J− 1 (a_(0N), a_(1N), . . . , a_(N) _(B) _(−1N) ₎

The values for x, y, are integers corresponding to pixels with

$\begin{pmatrix}x \\y\end{pmatrix} \in {\left\lbrack {0,1,{\ldots 31}} \right\rbrack^{2}.}$

The images may have a pixel size of 32×32. The dataset of beam profileimages may be generated by using the above formula for f_(k) incombination with a parameter set to obtain a continuous description off_(k). The values for each pixel in the 32×32-image may be obtained byinserting integer values from 0, . . . , 31 for x, y, in f_(k)(x,y). Forexample, for pixel (6,9), the value f_(k)(6,9) may be computed.

Subsequently, for each image f_(k), the feature value φ_(k)corresponding to the filter ϕ may be calculated,ϕ(f_(k)(x,y),z_(k))=φ_(k), wherein z_(k) is a distance valuecorresponding to the image f_(k)from the predefined data set. Thisyields a dataset with corresponding generated feature values φ_(k). Thehypothesis testing may use a Null-hypothesis that the filter does notdistinguish between material classifier. The Null-Hypothesis may begiven by H₀: μ₁=μ₂= . . . =μ_(j), wherein μ_(m) is the expectation valueof each material-group corresponding to the feature values φ_(k). Indexm denotes the material group. The hypothesis testing may use asalternative hypothesis that the filter does distinguish between at leasttwo material classifiers. The alternative hypothesis may be given by H₁:∃m,m′: μ_(m)≠μ_(m′). As used herein, the term “not distinguish betweenmaterial classifiers” refers to that the expectation values of thematerial classifiers are identical. As used herein, the term“distinguishes material classifiers” refers to that at least twoexpectation values of the material classifiers differ. As used herein“distinguishes at least two material classifiers” is used synonymous to“suitable material classifier”. The hypothesis testing may comprise atleast one analysis of variance (ANOVA) on the generated feature values.In particular, the hypothesis testing may comprise determining amean-value of the feature values for each of the J materials, i.e. intotal J mean values,

${{\overset{\_}{\varphi}}_{m} = \frac{{\sum}_{i}\varphi_{i,m}}{N_{m}}},$

for m∈[0,1, . . . , J−1], wherein N_(m) gives the number of featurevalues for each of the J materials in the predefined data set. Thehypothesis testing may comprise determining a mean-value of all Nfeature value

$\overset{\_}{\varphi} = {\frac{{\sum}_{m}{\sum}_{i}\varphi_{i,m}}{N}.}$

The hypothesis testing may comprise determining a Mean Sum Squareswithin:

mssw=(Σ_(m)Σ_(i)(φ_(i,m)−φ _(m))²)/(N−J).

The hypothesis testing may comprise determining a Mean Sum of Squaresbetween,

mssb=(Σ_(m)(φ _(m)−φ)² N _(m)/(J−1).

The hypothesis testing may comprise performing an F-Test:

${{\circ {{CDF}(x)}} = {I_{\frac{d_{1^{x}}}{{d_{1}x} + d_{2}}}\left( {\frac{d_{1}}{2},\frac{d_{2}}{2}} \right)}},{{{where}d_{1}} = {N - J}},{d_{2} = {J - 1}},$∘F(x) = 1 − CDF(x) ∘p = F(mssb/mssw)

Herein, I_(x) is the regularized incomplete Beta-Function,

${{I_{x}\left( {a,b} \right)} = \frac{B\left( {{x;a},b} \right)}{B\left( {a,b} \right)}},$

with the Euler Beta-Function B(a,b)=∫₀ ¹t^(a-1)(1−t)^(b-1)dt and B(x;a,b)=∫₀ ^(x)t^(a-1)(1−t)^(b-1)dt being the incomplete Beta-Function. Theimage filter may pass the hypothesis testing if a p-value, p, is smalleror equal than a pre-defined level of significance. The filter may passthe hypothesis testing if p≤0.075, preferably p≤0.05, more preferablyp≤0.025, and most preferably p≤0.01. For example, in case thepre-defined level of significance is α=0.075, the image filter may passthe hypothesis testing if the p-value is smaller than α=0.075. In thiscase the Null-hypothesis H₀ can be rejected and the alternativehypothesis H₁ can be accepted. The image filter thus distinguishes atleast two material classifiers. Thus, the image filter passes thehypothesis testing.

In the following, image filters are described assuming that thereflection image comprises at least one reflection feature, inparticular a spot image. A spot image f may be given by a function ƒ:

²→

_(≥0), wherein the background of the image f may be already subtracted.However, other reflection features may be possible.

For example, the material dependent image filter may be a luminancefilter. The luminance filter may return a luminance measure of a spot asmaterial feature. The material feature may be determined by

${\varphi_{2m} = {{\Phi\left( {f,z} \right)} = {- {\int{{f(x)}dx\frac{z^{2}}{d_{ray} \cdot n}}}}}},$

where f is the spot image. The distance of the spot is denoted by z,where z may be obtained for example by using a depth-from-defocus ordepth-from-photon ratio technique and/or by using a triangulationtechnique. The surface normal of the material is given by n∈

³ and can be obtained as the normal of the surface spanned by at leastthree measured points. The vector d_(ray)∈

³ is the direction vector of the light source. Since the position of thespot is known by using a depth-from-defocus or depth-from-photon ratiotechnique and/or by using a triangulation technique wherein the positionof the light source is known as a parameter of the detector system,d_(ray), is the difference vector between spot and light sourcepositions.

For example, the material dependent image filter may be a filter havingan output dependent on a spot shape. This material dependent imagefilter may return a value which correlates to the translucence of amaterial as material feature. The translucence of materials influencesthe shape of the spots. The material feature may be given by

${\varphi_{2m} = {{\Phi(f)} = \frac{\int{{H\left( {{f(x)} - {\alpha h}} \right)}dx}}{\int{{H\left( {{f(x)} - {\beta h}} \right)}dx}}}},$

wherein 0<α,β<1 are weights for the spot height h, and H denotes theHeavyside function, i.e. H(x)=1: x≥0, H(x)=0: x<0. The spot height h maybe determined by

h=∫ _(B) _(r) f(x)dx,

where B_(r) is an inner circle of a spot with radius r.

For example, the material dependent image filter may be a squared normgradient. This material dependent image filter may return a value whichcorrelates to a measure of soft and hard transitions and/or roughness ofa spot as material feature. The material feature may be defined by

φ_(2m)=Φ(f)=∫∥∇f(x)∥² dx.

For example, the material dependent image filter may be a standarddeviation. The standard deviation of the spot may be determined by

φ_(2m)=Φ(f)=∫(f(x)−μ)² dx,

Wherein μ is the mean value given by μ=f(f(x))dx.

For example, the material dependent image filter may be a smoothnessfilter such as a Gaussian filter or median filter. In one embodiment ofthe smoothness filter, this image filter may refer to the observationthat volume scattering exhibits less speckle contrast compared todiffuse scattering materials. This image filter may quantify thesmoothness of the spot corresponding to speckle contrast as materialfeature. The material feature may be determined by

${\varphi_{2m} = {{\Phi\left( {f,_{Z}} \right)} = {\frac{\int{{❘{{{\mathcal{F}(f)}(x)} - {f(x)}}❘}{dx}}}{\int{{f(x)}dx}} \cdot \frac{1}{z}}}},$

wherein

is a smoothness function, for example a median filter or Gaussianfilter. This image filter may comprise dividing by the distance z, asdescribed in the formula above. The distance z may be determined forexample using a depth-from-defocus or depth-from-photon ratio techniqueand/or by using a triangulation technique. This may allow the filter tobe insensitive to distance. In one embodiment of the smoothness filter,the smoothness filter may be based on the standard deviation of anextracted speckle noise pattern. A speckle noise pattern N can bedescribed in an empirical way by

f(x)=f ₀(x)·(N(X)+1),

where f₀ is an image of a despeckled spot. N(X) is the noise term thatmodels the speckle pattern. The computation of a despeckled image may bedifficult. Thus, the despeckled image may be approximated with asmoothed version of f, i.e. f₀≈

(f), wherein

is a smoothness operator like a Gaussian filter or median filter. Thus,an approximation of the speckle pattern may be given by

${N(X)} = {\frac{f(x)}{\mathcal{F}\left( {f(x)} \right)} - {1.}}$

The material feature of this filter may be determined by

${\varphi_{2m} = {{\Phi(f)} = \sqrt{{Var}\left( {\frac{f}{\mathcal{F}(f)} - 1} \right)}}},$

Wherein Var denotes the variance function.

For example, the image filter may be a grey-level-occurrence-basedcontrast filter. This material filter may be based on the grey leveloccurrence matrix M_(f,ρ)(g₁g₂)=[p_(g1,g2)], whereas p_(g1,g2) is theoccurrence rate of the grey combination (g₁,g₂)=[f(x₁,y₁),f(x₂,y₂)], andthe relation ρ defines the distance between (x₁,y₁) and (x₂,y₂), whichis ρ (x,y)=(x+a,y+b) with a and b selected from 0,1.

The material feature of the grey-level-occurrence-based contrast filtermay be given by

$\varphi_{2m} = {{\Phi(f)} = {\sum\limits_{i,{j = 0}}^{N - 1}{{p_{ij}\left( {i - j} \right)}^{2}.}}}$

For example, the image filter may be a grey-level-occurrence-basedenergy filter. This material filter is based on the grey leveloccurrence matrix defined above.

The material feature of the grey-level-occurrence-based energy filtermay be given by

$\varphi_{2m} = {{\Phi(f)} = {\sum\limits_{i,{j = 0}}^{N - 1}{\left( p_{ij} \right)^{2}.}}}$

For example, the image filter may be a grey-level-occurrence-basedhomogeneity filter. This material filter is based on the grey leveloccurrence matrix defined above.

The material feature of the grey-level-occurrence-based homogeneityfilter may be given by

$\varphi_{2m} = {{\Phi(f)} = {\sum\limits_{i,{j = 0}}^{N - 1}{\frac{p_{ij}}{1 + {❘{i - j}❘}}.}}}$

For example, the image filter may be a grey-level-occurrence-baseddissimilarity filter. This material filter is based on the grey leveloccurrence matrix defined above.

The material feature of the grey-level-occurrence-based dissimilarityfilter may be given by

$\varphi_{2m} = {{\Phi(f)} = {- {\sum\limits_{i,{j = 0}}^{N - 1}{\sqrt{p_{ij}\log\left( p_{ij} \right)}.}}}}$

For example, the image filter may be a Law's energy filter. Thismaterial filter may be based on the laws vector L₅=[1,4,6,4,1] andE₅=[−1,−2,0,−2,−1] and the matrices L₅(E₅)^(T) and E₅(L₅)^(T). The imagef_(k) is convoluted with these matrices:

${f_{k,{L5E5}}^{*}\left( {x,y} \right)} = {\sum\limits_{i - 2}^{2}{\sum\limits_{j - 2}^{2}{{f_{k}\left( {{x + i},{y + j}} \right)}{L_{5}\left( E_{5} \right)}^{T}}}}$and${f_{k,{E5L5}}^{*}\left( {x,y} \right)} = {{\sum}_{i - 2}^{2}{\sum}_{j - 2}^{2}{f_{k}\left( {{x + i},{y + j}} \right)}{{E_{5}\left( L_{5} \right)}^{T}.}}$${E = {\int{\frac{f_{k,{L5E5}}^{*}\left( {x,y} \right)}{\max\left( {f_{k,{L5E5}}^{*}\left( {x,y} \right)} \right)}dx{dy}}}},$${F = {\int{\frac{f_{k,{E5L5}}^{*}\left( {x,y} \right)}{\max\left( {f_{k,{E5L5}}^{*}\left( {x,y} \right)} \right)}dxdy}}},$

Whereas the material feature of Law's energy filter may be determined by

φ_(2m)=ϕ(f)=E/F.

For example, the material dependent image filter may be a threshold areafilter. This material feature may relate two areas in the image plane. Afirst area Ω1, may be an area wherein the function f is larger than αtimes the maximum of f. A second area Ω2, may be an area wherein thefunction f is smaller than α times the maximum of f, but larger than athreshold value ε times the maximum of f. Preferably α may be 0.5 and εmay be 0.05. Due to speckles or noise, the areas may not simplycorrespond to an inner and an outer circle around the spot center. As anexample, Ω1 may comprise speckles or unconnected areas in the outercircle. The material feature may be determined by

${\varphi_{2m} = {{\Phi(f)} = \frac{\int_{\Omega 1}1}{\int_{\Omega 2}1}}},$

wherein Ω1={x|f(x)>α·max(f(x))} and Ω2={x|ε·max(f(x))<f(x)<α·max(f(x))}.

The processing unit may be configured for using at least onepredetermined relationship between the material feature ϕ_(2m) and thematerial property of the surface having generated the reflection featurefor determining the material property of the surface having generatedthe reflection feature. The predetermined relationship may be one ormore of an empirical relationship, a semi-empiric relationship and ananalytically derived relationship. The processing unit may comprise atleast one data storage device for storing the predeterminedrelationship, such as a lookup list or a lookup table.

While feature based approaches, as the approaches described above, areaccurate enough to differentiate between skin and surface-onlyscattering materials, the differentiation between skin and carefullyselected attack materials, which involve volume scattering as well, ismore challenging. Step b) may comprise using artificial intelligence, inparticular convolutional neuronal networks. Using reflection images asinput for convolutional neuronal networks may enable the generation ofclassification models with sufficient accuracy to differentiate betweenskin and other volume-scattering materials. Since only physically validinformation is passed to the network by selecting important regions inthe reflection image, only compact training data sets may be needed.Additionally, very compact network architectures can be generated.

Specifically, in the skin detection step at least one parametrized skinclassification model may be used. The parametrized skin classificationmodel may be configured for classifying skin and other materials byusing the second image as an input. The skin classification model may beparametrized by using one or more of machine learning, deep learning,neural networks, or other form of artificial intelligence. The term“machine-learning” as used herein is a broad term and is to be given itsordinary and customary meaning to a person of ordinary skill in the artand is not to be limited to a special or customized meaning. The termspecifically may refer, without limitation, to a method of usingartificial intelligence (AI) for automatically model building, inparticular for parametrizing models. The term “skin classificationmodel” may refer to a classification model configured for discriminatinghuman skin from other materials. The property characteristic for skinmay be determined by applying an optimization algorithm in terms of atleast one optimization target on the skin classification model. Themachine learning may be based on at least one neuronal network, inparticular a convolutional neural network. Weights and/or topology ofthe neuronal network may be predetermined and/or pre-defined.Specifically, the training of the skin classification model may beperformed using machine-learning. The skin classification model maycomprise at least one machine-learning architecture and modelparameters. For example, the machine-learning architecture may be or maycomprise one or more of: linear regression, logistic regression, randomforest, naive Bayes classifications, nearest neighbors, neural networks,convolutional neural networks, generative adversarial networks, supportvector machines, or gradient boosting algorithms or the like. The term“training”, also denoted learning, as used herein, is a broad term andis to be given its ordinary and customary meaning to a person ofordinary skill in the art and is not to be limited to a special orcustomized meaning. The term specifically may refer, without limitation,to a process of building the skin classification model, in particulardetermining and/or updating parameters of the skin classification model.The skin classification model may be at least partially data-driven. Forexample, the skin classification model may be based on experimentaldata, such as data determined by illuminating a plurality of humans andartificial objects such as masks and recording the reflection pattern.For example, the training may comprise using at least one trainingdataset, wherein the training data set comprises images, in particularsecond images, of a plurality of humans and artificial objects withknown material property.

The skin detection step may comprises using at least one 2D face andfacial landmark detection algorithm configured for providing at leasttwo locations of characteristic points of a human face. For example, thelocations may be eye locations, forehead or cheeks. 2D face and faciallandmark detection algorithms may provide locations of characteristicpoints of a human face such as eye locations. Since there are subtledifferences in the reflection of the different zones of a face (forexample forehead or cheeks), region specific models can be trained. Inthe skin detection step, preferably at least one region specificparametrized skin classification model is used. The skin classificationmodel may comprise a plurality of region specific parametrized skinclassification models, such as for different regions, and/or the skinclassification model may be trained using region specific data such asby filtering the images used for training. For example, for training twodifferent regions may be used such as eye-cheek region to below the noseand, in particular in case not enough reflection features can beidentified within this region, the region of the forehead may be used.However, other regions may be possible, too.

The detected face is characterized as skin if the material propertycorresponds to at least one property characteristic for skin. Theprocessing unit may be configured for identifying a reflection featureas to be generated by illuminating biological tissue, in particularskin, in case its corresponding material property fulfills the at leastone predetermined or predefined criterion. The reflection feature may beidentified as to be generated by human skin in case the materialproperty indicates “human skin”. The reflection feature may beidentified as to be generated by human skin in case the materialproperty is within at least one threshold and/or at least one range. Theat least one threshold value and/or range may be stored in a table or alookup table and may be determined e.g. empirically and may, as anexample, be stored in at least one data storage device of the processingunit. The processing unit is configured for identifying the reflectionfeature as to be background otherwise. Thus, the processing unit may beconfigured for assigning each projected spot with a material property,e.g. skin yes or no.

The 3D detection step may be performed after the skin detection stepand/or the face detection step. However, other embodiments are feasible,in which the 3D detection step is performed before the skin detectionstep and/or the face detection step. The material property may bedetermined by evaluating ϕ_(2m) subsequently after determining of thelongitudinal coordinate z in step d) such that the information about thelongitudinal coordinate z can be considered for evaluating of ϕ_(2m).

The 3D detection step comprises determining the second beam profileinformation of at least four of the reflection features located insidethe image region of the second image corresponding to the image regionof the first image comprising the identified geometrical feature byanalysis of their beam profiles. The 3D detection step may comprisedetermining a second beam profile information of at least four of thereflection features by analysis of their respective beam profile. Thesecond beam profile information may comprise a quotient Q of areas ofthe beam profile.

As used herein, the term “analysis of the beam profile” may generallyrefer to evaluating of the beam profile and may comprise at least onemathematical operation and/or at least one comparison and/or at leastsymmetrizing and/or at least one filtering and/or at least onenormalizing. For example, the analysis of the beam profile may compriseat least one of a histogram analysis step, a calculation of a differencemeasure, application of a neural network, application of a machinelearning algorithm. The processing unit may be configured forsymmetrizing and/or for normalizing and/or for filtering the beamprofile, in particular to remove noise or asymmetries from recordingunder larger angles, recording edges or the like. The processing unitmay filter the beam profile by removing high spatial frequencies such asby spatial frequency analysis and/or median filtering or the like.Summarization may be performed by center of intensity of the light spotand averaging all intensities at the same distance to the center. Theprocessing unit may be configured for normalizing the beam profile to amaximum intensity, in particular to account for intensity differencesdue to the recorded distance. The processing unit may be configured forremoving influences from background light from the beam profile, forexample, by an imaging without illumination.

The processing unit may be configured for determining at least onelongitudinal coordinate z_(DPR) for reflection features located insidethe image region of the second image corresponding to the image regionof the first image comprising the identified geometrical feature byanalysis of the beam profile of the respective reflection feature. Theprocessing unit may be configured for determining the longitudinalcoordinate z_(DPR) for the reflection features by using the so calleddepth-from-photon-ratio technique, also denoted as beam profileanalysis. With respect to depth-from-photon-ratio (DPR) techniquereference is made to WO 2018/091649 A1, WO 2018/091638 A1 and WO2018/091640 A1, the full content of which is included by reference.

The processing unit may be configured for determining at least one firstarea and at least one second area of the reflection beam profile of eachof the reflection features and/or of the reflection features in at leastone region of interest. The processing unit is configured forintegrating the first area and the second area.

The analysis of the beam profile of one of the reflection features maycomprise determining at least one first area and at least one secondarea of the beam profile. The first area of the beam profile may be anarea A1 and the second area of the beam profile may be an area A2. Theprocessing unit may be configured for integrating the first area and thesecond area. The processing unit may be configured to derive a combinedsignal, in particular a quotient Q, by one or more of dividing theintegrated first area and the integrated second area, dividing multiplesof the integrated first area and the integrated second area, dividinglinear combinations of the integrated first area and the integratedsecond area. The processing unit may configured for determining at leasttwo areas of the beam profile and/or to segment the beam profile in atleast two segments comprising different areas of the beam profile,wherein overlapping of the areas may be possible as long as the areasare not congruent. For example, the processing unit may be configuredfor determining a plurality of areas such as two, three, four, five, orup to ten areas. The processing unit may be configured for segmentingthe light spot into at least two areas of the beam profile and/or tosegment the beam profile in at least two segments comprising differentareas of the beam profile. The processing unit may be configured fordetermining for at least two of the areas an integral of the beamprofile over the respective area. The processing unit may be configuredfor comparing at least two of the determined integrals. Specifically,the processing unit may be configured for determining at least one firstarea and at least one second area of the beam profile. As used herein,the term “area of the beam profile” generally refers to an arbitraryregion of the beam profile at the position of the optical sensor usedfor determining the quotient Q. The first area of the beam profile andthe second area of the beam profile may be one or both of adjacent oroverlapping regions. The first area of the beam profile and the secondarea of the beam profile may be not congruent in area. For example, theprocessing unit may be configured for dividing a sensor region of theCMOS sensor into at least two sub-regions, wherein the processing unitmay be configured for dividing the sensor region of the CMOS sensor intoat least one left part and at least one right part and/or at least oneupper part and at least one lower part and/or at least one inner and atleast one outer part. Additionally or alternatively, the camera maycomprise at least two optical sensors, wherein the light-sensitive areasof a first optical sensor and of a second optical sensor may be arrangedsuch that the first optical sensor is adapted to determine the firstarea of the beam profile of the reflection feature and that the secondoptical sensor is adapted to determine the second area of the beamprofile of the reflection feature. The processing unit may be adapted tointegrate the first area and the second area. The processing unit may beconfigured for using at least one predetermined relationship between thequotient Q and the longitudinal coordinate for determining thelongitudinal coordinate. The predetermined relationship may be one ormore of an empiric relationship, a semi-empiric relationship and ananalytically derived relationship. The processing unit may comprise atleast one data storage device for storing the predeterminedrelationship, such as a lookup list or a lookup table.

The first area of the beam profile may comprise essentially edgeinformation of the beam profile and the second area of the beam profilecomprises essentially center information of the beam profile, and/or thefirst area of the beam profile may comprise essentially informationabout a left part of the beam profile and the second area of the beamprofile comprises essentially information about a right part of the beamprofile. The beam profile may have a center, i.e. a maximum value of thebeam profile and/or a center point of a plateau of the beam profileand/or a geometrical center of the light spot, and falling edgesextending from the center. The second region may comprise inner regionsof the cross section and the first region may comprise outer regions ofthe cross section. As used herein, the term “essentially centerinformation” generally refers to a low proportion of edge information,i.e. proportion of the intensity distribution corresponding to edges,compared to a proportion of the center information, i.e. proportion ofthe intensity distribution corresponding to the center. Preferably, thecenter information has a proportion of edge information of less than10%, more preferably of less than 5%, most preferably the centerinformation comprises no edge content. As used herein, the term“essentially edge information” generally refers to a low proportion ofcenter information compared to a proportion of the edge information. Theedge information may comprise information of the whole beam profile, inparticular from center and edge regions. The edge information may have aproportion of center information of less than 10%, preferably of lessthan 5%, more preferably the edge information comprises no centercontent. At least one area of the beam profile may be determined and/orselected as second area of the beam profile if it is close or around thecenter and comprises essentially center information. At least one areaof the beam profile may be determined and/or selected as first area ofthe beam profile if it comprises at least parts of the falling edges ofthe cross section. For example, the whole area of the cross section maybe determined as first region.

Other selections of the first area A1 and second area A2 may befeasible. For example, the first area may comprise essentially outerregions of the beam profile and the second area may comprise essentiallyinner regions of the beam profile. For example, in case of atwo-dimensional beam profile, the beam profile may be divided in a leftpart and a right part, wherein the first area may comprise essentiallyareas of the left part of the beam profile and the second area maycomprise essentially areas of the right part of the beam profile.

The edge information may comprise information relating to a number ofphotons in the first area of the beam profile and the center informationmay comprise information relating to a number of photons in the secondarea of the beam profile. The processing unit may be configured fordetermining an area integral of the beam profile. The processing unitmay be configured for determining the edge information by integratingand/or summing of the first area. The processing unit may be configuredfor determining the center information by integrating and/or summing ofthe second area. For example, the beam profile may be a trapezoid beamprofile and the processing unit may be configured for determining anintegral of the trapezoid. Further, when trapezoid beam profiles may beassumed, the determination of edge and center signals may be replaced byequivalent evaluations making use of properties of the trapezoid beamprofile such as determination of the slope and position of the edges andof the height of the central plateau and deriving edge and centersignals by geometric considerations.

In one embodiment, A1 may correspond to a full or complete area of afeature point on the optical sensor. A2 may be a central area of thefeature point on the optical sensor. The central area may be a constantvalue. The central area may be smaller compared to the full area of thefeature point. For example, in case of a circular feature point, thecentral area may have a radius from 0.1 to 0.9 of a full radius of thefeature point, preferably from 0.4 to 0.6 of the full radius.

In one embodiment, the illumination pattern may comprise at least oneline pattern. A1 may correspond to an area with a full line width of theline pattern on the optical sensors, in particular on the lightsensitive area of the optical sensors. The line pattern on the opticalsensor may be widened and/or displaced compared to the line pattern ofthe illumination pattern such that the line width on the optical sensorsis increased. In particular, in case of a matrix of optical sensors, theline width of the line pattern on the optical sensors may change fromone column to another column. A2 may be a central area of the linepattern on the optical sensor. The line width of the central area may bea constant value, and may in particular correspond to the line width inthe illumination pattern. The central area may have a smaller line widthcompared to the full line width. For example, the central area may havea line width from 0.1 to 0.9 of the full line width, preferably from 0.4to 0.6 of the full line width. The line pattern may be segmented on theoptical sensors. Each column of the matrix of optical sensors maycomprise center information of intensity in the central area of the linepattern and edge information of intensity from regions extending furtheroutwards from the central area to edge regions of the line pattern.

In one embodiment, the illumination pattern may comprise at least pointpattern. A1 may correspond to an area with a full radius of a point ofthe point pattern on the optical sensors. A2 may be a central area ofthe point in the point pattern on the optical sensors. The central areamay be a constant value. The central area may have a radius compared tothe full radius. For example, the central area may have a radius from0.1 to 0.9 of the full radius, preferably from 0.4 to 0.6 of the fullradius.

The illumination pattern may comprise both at least one point patternand at least one line pattern. Other embodiments in addition oralternatively to line pattern and point pattern are feasible.

The processing unit may be configured to derive a quotient Q by one ormore of dividing the integrated first area and the integrated secondarea, dividing multiples of the integrated first area and the integratedsecond area, dividing linear combinations of the integrated first areaand the integrated second area.

The processing unit may be configured to derive the quotient Q by one ormore of dividing the first area and the second area, dividing multiplesof the first area and the second area, dividing linear combinations ofthe first area and the second area. The processing unit may beconfigured for deriving the quotient Q by

$Q = \frac{\int{\int_{A1}{{E\left( {x,y} \right)}{dxdy}}}}{\int{\int_{A2}{{E\left( {x,y} \right)}{dxdy}}}}$

wherein x and y are transversal coordinates, A1 and A2 are the first andsecond area of the beam profile, respectively, and E(x,y) denotes thebeam profile.

Additionally or alternatively, the processing unit may be adapted todetermine one or both of center information or edge information from atleast one slice or cut of the light spot. This may be realized, forexample, by replacing the area integrals in the quotient Q by a lineintegral along the slice or cut. For improved accuracy, several slicesor cuts through the light spot may be used and averaged. In case of anelliptical spot profile, averaging over several slices or cuts mayresult in improved distance information.

For example, in case of the optical sensor having a matrix of pixels,the processing unit may be configured for evaluating the beam profile,by

-   -   determining the pixel having the highest sensor signal and        forming at least one center signal;    -   evaluating sensor signals of the matrix and forming at least one        sum signal;    -   determining the quotient Q by combining the center signal and        the sum signal; and    -   determining at least one longitudinal coordinate z of the object        by evaluating the quotient Q.

As used herein, a “sensor signal” generally refers to a signal generatedby the optical sensor and/or at least one pixel of the optical sensor inresponse to illumination. Specifically, the sensor signal may be or maycomprise at least one electrical signal, such as at least one analogueelectrical signal and/or at least one digital electrical signal. Morespecifically, the sensor signal may be or may comprise at least onevoltage signal and/or at least one current signal. More specifically,the sensor signal may comprise at least one photocurrent. Further,either raw sensor signals may be used, or the display device, theoptical sensor or any other element may be adapted to process orpreprocess the sensor signal, thereby generating secondary sensorsignals, which may also be used as sensor signals, such as preprocessingby filtering or the like. The term “center signal” generally refers tothe at least one sensor signal comprising essentially center informationof the beam profile. As used herein, the term “highest sensor signal”refers to one or both of a local maximum or a maximum in a region ofinterest. For example, the center signal may be the signal of the pixelhaving the highest sensor signal out of the plurality of sensor signalsgenerated by the pixels of the entire matrix or of a region of interestwithin the matrix, wherein the region of interest may be predeterminedor determinable within an image generated by the pixels of the matrix.The center signal may arise from a single pixel or from a group ofoptical sensors, wherein, in the latter case, as an example, the sensorsignals of the group of pixels may be added up, integrated or averaged,in order to determine the center signal. The group of pixels from whichthe center signal arises may be a group of neighboring pixels, such aspixels having less than a predetermined distance from the actual pixelhaving the highest sensor signal, or may be a group of pixels generatingsensor signals being within a predetermined range from the highestsensor signal. The group of pixels from which the center signal arisesmay be chosen as large as possible in order to allow maximum dynamicrange. The processing unit may be adapted to determine the center signalby integration of the plurality of sensor signals, for example theplurality of pixels around the pixel having the highest sensor signal.For example, the beam profile may be a trapezoid beam profile and theprocessing unit may be adapted to determine an integral of thetrapezoid, in particular of a plateau of the trapezoid.

As outlined above, the center signal generally may be a single sensorsignal, such as a sensor signal from the pixel in the center of thelight spot, or may be a combination of a plurality of sensor signals,such as a combination of sensor signals arising from pixels in thecenter of the light spot, or a secondary sensor signal derived byprocessing a sensor signal derived by one or more of the aforementionedpossibilities. The determination of the center signal may be performedelectronically, since a comparison of sensor signals is fairly simplyimplemented by conventional electronics, or may be performed fully orpartially by software. Specifically, the center signal may be selectedfrom the group consisting of: the highest sensor signal; an average of agroup of sensor signals being within a predetermined range of tolerancefrom the highest sensor signal; an average of sensor signals from agroup of pixels containing the pixel having the highest sensor signaland a predetermined group of neighboring pixels; a sum of sensor signalsfrom a group of pixels containing the pixel having the highest sensorsignal and a predetermined group of neighboring pixels; a sum of a groupof sensor signals being within a predetermined range of tolerance fromthe highest sensor signal; an average of a group of sensor signals beingabove a predetermined threshold; a sum of a group of sensor signalsbeing above a predetermined threshold; an integral of sensor signalsfrom a group of optical sensors containing the optical sensor having thehighest sensor signal and a predetermined group of neighboring pixels;an integral of a group of sensor signals being within a predeterminedrange of tolerance from the highest sensor signal; an integral of agroup of sensor signals being above a predetermined threshold.

Similarly, the term “sum signal” generally refers to a signal comprisingessentially edge information of the beam profile. For example, the sumsignal may be derived by adding up the sensor signals, integrating overthe sensor signals or averaging over the sensor signals of the entirematrix or of a region of interest within the matrix, wherein the regionof interest may be predetermined or determinable within an imagegenerated by the optical sensors of the matrix. When adding up,integrating over or averaging over the sensor signals, the actualoptical sensors from which the sensor signal is generated may be leftout of the adding, integration or averaging or, alternatively, may beincluded into the adding, integration or averaging. The processing unitmay be adapted to determine the sum signal by integrating signals of theentire matrix, or of the region of interest within the matrix. Forexample, the beam profile may be a trapezoid beam profile and theprocessing unit may be adapted to determine an integral of the entiretrapezoid. Further, when trapezoid beam profiles may be assumed, thedetermination of edge and center signals may be replaced by equivalentevaluations making use of properties of the trapezoid beam profile suchas determination of the slope and position of the edges and of theheight of the central plateau and deriving edge and center signals bygeometric considerations.

Similarly, the center signal and edge signal may also be determined byusing segments of the beam profile such as circular segments of the beamprofile. For example, the beam profile may be divided into two segmentsby a secant or a chord that does not pass the center of the beamprofile. Thus, one segment will essentially contain edge information,while the other segment will contain essentially center information. Forexample, to further reduce the amount of edge information in the centersignal, the edge signal may further be subtracted from the centersignal.

The quotient Q may be a signal which is generated by combining thecenter signal and the sum signal. Specifically, the determining mayinclude one or more of: forming a quotient of the center signal and thesum signal or vice versa; forming a quotient of a multiple of the centersignal and a multiple of the sum signal or vice versa; forming aquotient of a linear combination of the center signal and a linearcombination of the sum signal or vice versa. Additionally oralternatively, the quotient Q may comprise an arbitrary signal or signalcombination which contains at least one item of information on acomparison between the center signal and the sum signal.

As used herein, the term “longitudinal coordinate for the reflectionfeature” refers to a distance between the optical sensor and the pointof the scene remitting the corresponding illumination features. Theprocessing unit may be configured for using the at least onepredetermined relationship between the quotient Q and the longitudinalcoordinate for determining the longitudinal coordinate. Thepredetermined relationship may be one or more of an empiricrelationship, a semi-empiric relationship and an analytically derivedrelationship. The processing unit may comprise at least one data storagedevice for storing the predetermined relationship, such as a lookup listor a lookup table.

The processing unit may be configured for executing at least onedepth-from-photon-ratio algorithm which computes distances for allreflection features with zero order and higher order.

The 3D detection step may comprise determining the at least one depthlevel from the second beam profile information of said reflectionfeatures by using the processing unit.

The processing unit may be configured for determining the depth map ofat least parts of the scene by determining at least one depthinformation of the reflection features located inside the image regionof the second image corresponding to the image region of the first imagecomprising the identified geometrical feature. As used herein, the term“depth” or depth information may refer to a distance between the objectand the optical sensor and may be given by the longitudinal coordinate.As used herein, the term “depth map” may re-fer to spatial distributionof depth. The processing unit may be configured for determining thedepth information of the reflection features by one or more of thefollowing techniques: depth-from-photon-ratio, structured light, beamprofile analysis, time-of-flight, shape-frommotion, depth-from-focus,triangulation, depth-from-defocus, stereo sensors. The depth map may bea thinly filled depth map comprising a few entries. Alternatively, thedepth may be crowded comprising a large amount of entries.

The detected face is characterized as 3D object if the depth leveldeviates from a pre-determined or pre-defined depth level of planeobjects. Step c) may comprise using 3D topology data of the face infront of the camera. The method may comprise determining a curvaturefrom the at least four of the reflection features located inside theimage region of the second image corresponding to the image region ofthe first image comprising the identified geometrical feature. Themethod may comprise comparing the curvature determined from the at leastfour of the reflection features to the pre-determined or pre-defineddepth level of plane objects. If the curvature exceeds an assumedcurvature of plane object the detected face may be characterized as 3Dobject, otherwise as plane object. The pre-determined or pre-defineddepth level of plane objects may be stored in at least one data storageof the processing unit such as a lookup list or a lookup table. Thepre-determined or predefined level of plane objects may beexperimentally determined and/or may be a theoretical level of planeobjects. The pre-determined or pre-defined depth level of plane objectsmay be at least one limit for at least one curvature and/or a range forat least one curvature.

3D features determined step c) may allow differentiating between highquality photographs and a 3D face-like structure. The combination ofsteps b) and c) may allow strengthening reliability of theauthentication with respect to attacks. 3D features can be combined withmaterial features to increase the security level. Since the samecomputational pipeline can be used to generate the input data for theskin classification and the generation of the 3D point cloud, bothproperties can be calculated from the same frame with low computationaleffort.

Preferably subsequent to steps a) to c) the authentication step may beperformed. The authentication step may be performed partially after eachof steps a) to c). The authentication may be aborted in case in step a)no face is detected and/or in step b) the reflection features isdetermined not to be generated by skin and/or in step c) the depth maprefers to a plane object. The authentication step comprisesauthenticating the detected face by using at least one authenticationunit if in step b) the detected face is characterized as skin and instep c) the detected face is characterized as 3D object.

Steps a) to d) may be performed by using at least one device, forexample at least one mobile device such as a mobile phone, smartphoneand the like, wherein access of the device is secured by using faceauthentication. Other devices may be possible, too such as an accesscontrol device controlling access to buildings, machines, automobilesand the like. The method may comprise permitting access to the device ifthe detected face is authenticated.

The method may comprise at least one enrollment step. In the enrollmentstep a user of the device may be enrolled. As used herein, the term“enrolling” may refer to a process of registering and/or signing upand/or teach in of a user for subsequent usage of the device. Usually,enrolling may be performed at first use of the device and/or forinitiating the device. However, embodiments are feasible in which aplurality of users may be enrolled, e.g. successively, such that theenrolling may be performed and/or repeated at an arbitrary time duringusage of the device. The enrolling may comprise generating a useraccount and/or user profile. The enrolling may comprise entering andstoring user data, in particular image data, via at least one userinterface. Specifically, at least one 2D image of the user is stored inat least one database. The enrollment step may comprise imaging at leastone image of the user, in particular a plurality of images. The imagesmay be recorded from different direction and/or the user may change hisorientation. Additionally, the enrollment step may comprise generatingat least one 3D image and/or a depth map for the user which may be usedin step d) for comparison. The database may a database of the device,e.g. of the processing unit, and/or may be an external database such asa cloud. The method comprises identifying the user by comparing the 2Dimage of the user with the first image. The method according to thepresent invention may allow significantly improving the presentationattack detection capabilities of biometric authentication methods. Inorder improve the overall authentication, person specific materialfingerprints as well as 3D topological features may be stored during theenrollment process in addition to the 2D image of the user. This mayallow a multifactor authentication within one device by using 2D, 3D andmaterial-derived features.

The method according to the present invention using beam profileanalysis technology may provide a concept to reliably detect human skinby analyzing reflections of laser spots, in particular in the NIRregime, on a face and distinguish it from reflections coming from attackmaterials that were produced to mimic a face. Additionally, beam profileanalysis simultaneously provides depth information by analyzing the samecamera frame. Therefore, 3D as well as skin security features may beprovided by the exact same technology.

Since also the 2D image of the face can be recorded by simply switchingoff the laser illumination, a fully secure face recognition pipeline canbe established solving the above-stated problem.

The reflection properties of human skin with respect to ethnic originbecome more similar when the laser wavelength is shifted towards the NIRregime. At a wavelength of 940 nm differences are at a minimum.Accordingly, different ethnic origins do not play a role for skinauthentication.

No time-consuming analysis of a series of frames may be necessary sincepresentation attack detection (via skin classification) is provided byjust one frame. A time frame for performing the complete method may be≤500 ms, preferably ≤250 ms. However, embodiments may be feasible inwhich the skin detection may be performed using a plurality of frames.Depending on confidence for identifying reflection features in thesecond image, and speed of the method, the method may comprise samplingreflection features over several frames in order to reach a more stableclassification.

Besides accuracy, also execution speed and power consumption areimportant requirements. Further restrictions on the availability ofcomputational resources can be introduced by security considerations.For example, steps a) to d) may have run in a secure zone of theprocessing unit to avoid any software-based manipulations during programexecution. The compact nature of the above described material detectionnetworks may solve this problem by showing excellent runtime behavior inthe said secure zone, whereas traditional PAD solutions require theexamination of several consecutive frames which results in largecomputational cost and longer response times of the algorithm.

In a further aspect of the present invention a computer program for faceauthentication configured for causing a computer or a computer networkto fully or partially perform the method according to the presentinvention, when executed on the computer or the computer network,wherein the computer program is configured for performing and/orexecuting at least steps a) to d) of the method according to the presentinvention. Specifically, the computer program may be stored on acomputer-readable data carrier and/or on a computer-readable storagemedium.

As used herein, the terms “computer-readable data carrier” and“computer-readable storage medium” specifically may refer tonon-transitory data storage means, such as a hardware storage mediumhaving stored thereon computer-executable instructions. Thecomputer-readable data carrier or storage medium specifically may be ormay comprise a storage medium such as a random-access memory (RAM)and/or a read-only memory (ROM).

Thus, specifically, one, more than one or even all of method steps asindicated above may be performed by using a computer or a computernetwork, preferably by using a computer program.

In a further aspect a computer-readable storage medium comprisinginstructions which, when executed by a computer or computer network,cause to carry out at least steps a) to d) of the method according tothe present invention.

Further disclosed and proposed herein is a data carrier having a datastructure stored thereon, which, after loading into a computer orcomputer network, such as into a working memory or main memory of thecomputer or computer network, may execute the method according to one ormore of the embodiments disclosed herein.

Further disclosed and proposed herein is a computer program product withprogram code means stored on a machine-readable carrier, in order toperform the method according to one or more of the embodiments disclosedherein, when the program is executed on a computer or computer network.As used herein, a computer program product refers to the program as atradable product. The product may generally exist in an arbitraryformat, such as in a paper format, or on a computer-readable datacarrier and/or on a computer-readable storage medium. Specifically, thecomputer program product may be distributed over a data network.

Finally, disclosed and proposed herein is a modulated data signal whichcontains instructions readable by a computer system or computer network,for performing the method according to one or more of the embodimentsdisclosed herein.

Referring to the computer-implemented aspects of the invention, one ormore of the method steps or even all of the method steps of the methodsaccording to one or more of the embodiments disclosed herein may beperformed by using a computer or computer network. Thus, generally, anyof the method steps including provision and/or manipulation of data maybe performed by using a computer or computer network. Generally, thesemethod steps may include any of the method steps, typically except formethod steps requiring manual work.

Specifically, further disclosed herein are:

-   -   a computer or computer network comprising at least one        processor, wherein the processor is adapted to perform the        method according to one of the embodiments described in this        description,    -   a computer loadable data structure that is adapted to perform        the method according to one of the embodiments described in this        description while the data structure is being executed on a        computer,    -   a computer program, wherein the computer program is adapted to        perform the method according to one of the embodiments described        in this description while the program is being executed on a        computer,    -   a computer program comprising program means for performing the        method according to one of the embodiments described in this        description while the computer program is being executed on a        computer or on a computer network,    -   a computer program comprising program means according to the        preceding embodiment, wherein the program means are stored on a        storage medium readable to a computer,    -   a storage medium, wherein a data structure is stored on the        storage medium and wherein the data structure is adapted to        perform the methods according to one of the embodiments        described in this description after having been loaded into a        main and/or working storage of a computer or of a computer        network, and    -   a computer program product having program code means, wherein        the program code means can be stored or are stored on a storage        medium, for performing the method according to one of the        embodiments described in this description, if the program code        means are executed on a computer or on a computer network.

In a further aspect a mobile device comprising at least one camera, atleast one illumination unit and at least one processing unit isdisclosed. The mobile device is configured for performing at least stepsa) to c), and optionally step d) of the method for face authenticationaccording to the present invention. Step d) may be performed by using atleast one authentication unit. The authentication unit may be a unit ofthe mobile device or may be an external authentication unit. Withrespect to definitions and embodiments of the mobile device reference ismade to definitions and embodiments described with respect to themethod.

In a further aspect of the present invention, use of the methodaccording to the present invention, such as according to one or more ofthe embodiments given above or given in further detail below, isproposed, for a purpose of use for biometric presentation attackdetection.

Overall, in the context of the present invention, the followingembodiments are regarded as preferred:

Embodiment 1 Method for face authentication comprising the followingsteps:

-   -   a) at least one face detection step, wherein the face detection        step comprises determining at least one first image by using at        least one camera, wherein the first image comprises at least one        two-dimensional image of a scene suspected to comprise the face,        wherein the face detection step comprises detecting the face in        the first image by identifying in the first image at least one        pre-defined or predetermined geometrical feature characteristic        for faces by using at least one processing unit;    -   b) at least one skin detection step, wherein the skin detection        step comprises projecting at least one illumination pattern        comprising a plurality of illumination features on the scene by        using at least one illumination unit and determining at least        one second image using the at least one camera, wherein the        second image comprises a plurality of reflection features        generated by the scene in response to illumination by the        illumination features, wherein each of the reflection features        comprises at least one beam profile, wherein the skin detection        step comprises determining a first beam profile information of        at least one of the reflection features located inside an image        region of the second image corresponding to an image region of        the first image comprising the identified geometrical feature by        analysis of its beam profile and determining at least one        material property of the reflection feature from the first beam        profile information by using the processing unit, wherein the        detected face is characterized as skin if the material property        corresponds to at least one property characteristic for skin;    -   c) at least one 3D detection step, wherein the 3D detection step        comprises determining a second beam profile information of at        least four of the reflection features located inside the image        region of the second image corresponding to the image region of        the first image comprising the identified geometrical feature by        analysis of their beam profiles and determining at least one        depth level from the second beam profile information of said        reflection features by using the processing unit, wherein the        detected face is characterized as 3D object if the depth level        deviates from a pre-determined or pre-defined depth level of        plane objects;    -   d) at least one authentication step, wherein the authentication        step comprises authenticating the detected face by using at        least one authentication unit if in step b) the detected face is        characterized as skin and in step c) the detected face is        characterized as 3D object.

Embodiment 2 The method according to the preceding embodiment, whereinsteps a) to d) are performed by using at least one device, whereinaccess of the device is secured by using face authentication, whereinthe method comprises permitting access to the device if the detectedface is authenticated.

Embodiment 3 The method according to the preceding embodiment, whereinthe method comprises at least one enrollment step, wherein in theenrollment step a user of the device is enrolled, wherein at least one2D image of the user is stored in at least one database, wherein themethod comprises identifying the user by comparing the 2D image of theuser with the first image.

Embodiment 4 The method according to any one of the precedingembodiments, wherein in the skin detection step at least oneparametrized skin classification model is used, wherein the parametrizedskin classification model is configured for classifying skin and othermaterials by using the second image as an input.

Embodiment 5 The method according to the preceding embodiment, whereinthe skin classification model is parametrized by using machine learning,wherein the property characteristic for skin is determined by applyingan optimization algorithm in terms of at least one optimization targeton the skin classification model.

Embodiment 6 The method according to any one of the two precedingembodiments, wherein the skin detection step comprises using at leastone 2D face and facial landmark detection algorithm configured forproviding at least two locations of characteristic points of a humanface, wherein in the skin detection step at least one region specificparametrized skin classification model is used.

Embodiment 7 The method according to any one of the precedingembodiments, wherein the illumination pattern comprises a periodic gridof laser spots.

Embodiment 8 The method according to any one of the precedingembodiments, wherein the illumination features have wavelengths in anear infrared (NIR) regime.

Embodiment 9 The method according to the preceding embodiment, whereinthe illumination features have wavelengths of 940 nm.

Embodiment 10 The method according to any one of the precedingembodiments, wherein a plurality of second images is determined, whereinthe reflection features of the plurality of second images are used forskin detection in step b) and/or for 3D detection in step c).

Embodiment 11 The method according to any one of the precedingembodiments, wherein the camera is or comprises at least one nearinfrared camera.

Embodiment 12 Computer program for face authentication configured forcausing a computer or a computer network to fully or partially performthe method according to any one of the preceding embodiments, whenexecuted on the computer or the computer network, wherein the computerprogram is configured for performing and/or executing at least steps a)to d) of the method according to any one of the preceding embodiments.

Embodiment 13 A computer-readable storage medium comprising instructionswhich, when executed by a computer or computer network, cause to carryout at least steps a) to d) of the method according to any one of thepreceding embodiments referring to a method.

Embodiment 14 A mobile device comprising at least one camera, at leastone illumination unit and at least one processing unit, the mobiledevice being configured for performing at least steps a) to c), andoptionally step d), of the method for face authentication according toany one of the preceding embodiments referring to a method.

Embodiment 15 Use of the method according to any one of the precedingclaims for biometric presentation attack detection.

BRIEF DESCRIPTION OF THE FIGURES

Further optional details and features of the invention are evident fromthe description of preferred exemplary embodiments which follows inconjunction with the dependent claims. In this context, the particularfeatures may be implemented in an isolated fashion or in combinationwith other features. The invention is not restricted to the exemplaryembodiments. The exemplary embodiments are shown schematically in thefigures. Identical reference numerals in the individual figures refer toidentical elements or elements with identical function, or elementswhich correspond to one another with regard to their functions.

Specifically, in the figures:

FIG. 1 shows an embodiment of a method for face authentication accordingto the present invention;

FIG. 2 shows an embodiment of a mobile device according to the presentinvention; and

FIG. 3 shows experimental results.

DETAILED DESCRIPTION OF THE EMBODIMENTS

FIG. 1 shows a flow chart of a method for face authentication accordingto the present invention. The face authentication may comprise verifyinga recognized object or part of the recognized object as human face.Specifically, the authentication may comprise distinguishing a realhuman face from attack materials that were produced to mimic a face. Theauthentication may comprise verifying identity of a respective userand/or assigning identity to a user. The authentication may comprisegenerating and/or providing identity information, e.g. to other devicessuch as to at least one authorization device for authorization foraccess of a mobile device, a machine, an automobile, a building or thelike. The identify information may be proofed by the authentication. Forexample, the identity information may be and/or may comprise at leastone identity token. In case of successful authentication the recognizedobject or part of the recognized object is verified to be a real faceand/or the identity of the object, in particular a user, is verified.

The method comprises the following steps:

-   -   a) (reference number 110) at least one face detection step,        wherein the face detection step comprises determining at least        one first image by using at least one camera 112, wherein the        first image comprises at least one two-dimensional image of a        scene suspected to comprise the face, wherein the face detection        step comprises detecting the face in the first image by        identifying in the first image at least one pre-defined or        predetermined geometrical feature characteristic for faces by        using at least one processing unit 114;    -   b) (reference number 116) at least one skin detection step,        wherein the skin detection step comprises projecting at least        one illumination pattern comprising a plurality of illumination        features on the scene by using at least one illumination unit        118 and determining at least one second image using the at least        one camera 112, wherein the second image comprises a plurality        of reflection features generated by the scene in response to        illumination by the illumination features, wherein each of the        reflection features comprises at least one beam profile, wherein        the skin detection step comprises determining a first beam        profile information of at least one of the reflection features        located inside an image region of the second image corresponding        to an image region of the first image comprising the identified        geometrical feature by analysis of its beam profile and        determining at least one material property of the reflection        feature from the first beam profile information by using the        processing unit 114, wherein the detected face is characterized        as skin if the material property corresponds to at least one        property characteristic for skin;    -   c) (reference number 120) at least one 3D detection step,        wherein the 3D detection step comprises determining a second        beam profile information of at least four of the reflection        features located inside the image region of the second image        corresponding to the image region of the first image comprising        the identified geometrical feature by analysis of their beam        profiles and determining at least one depth level from the        second beam profile information of said reflection features by        using the processing unit 114, wherein the detected face is        characterized as 3D object if the depth level deviates from a        pre-determined or pre-defined depth level of plane objects;    -   d) (reference number 122) at least one authentication step,        wherein the authentication step comprises authenticating the        detected face by using at least one authentication unit if in        step b) the detected face is characterized as skin and in        step c) the detected face is characterized as 3D object.

The method steps may be performed in the given order or may be performedin a different order. Further, one or more additional method steps maybe present which are not listed. Further, one, more than one or even allof the method steps may be performed repeatedly.

The camera 112 may comprise at least one imaging element configured forrecording or capturing spatially resolved one-dimensional,two-dimensional or even three-dimensional optical data or information.The camera 112 may be a digital camera. As an example, the camera 112may comprise at least one camera chip, such as at least one CCD chipand/or at least one CMOS chip configured for recording images. Thecamera 112 may be or may comprise at least one near infrared camera. Theimage may relate to data recorded by using the camera 112, such as aplurality of electronic readings from the imaging device, such as thepixels of the camera chip. The camera 112, besides the at least onecamera chip or imaging chip, may comprise further elements, such as oneor more optical elements, e.g. one or more lenses. As an example, thecamera 112 may be a fix-focus camera, having at least one lens which isfixedly adjusted with respect to the camera. Alternatively, however, thecamera 112 may also comprise one or more variable lenses which may beadjusted, automatically or manually.

The camera 112 may be a camera of a mobile device 124 such as ofnotebook computers, tablets or, specifically, cell phones such as smartphones and the like. Thus, specifically, the camera 112 may be part ofthe mobile device 124 which, besides the at least one camera 112,comprises one or more data processing devices such as one or more dataprocessors. Other cameras, however, are feasible. The mobile device 124may be a mobile electronics device, more specifically to a mobilecommunication device such as a cell phone or smart phone. Additionallyor alternatively, the mobile device 124 may also refer to a tabletcomputer or another type of portable computer. An embodiment of a mobiledevice according to the present invention is shown in FIG. 2 .

Specifically, the camera 112 may be or may comprise at least one opticalsensor 126 having at least one light-sensitive area. The optical sensor126 specifically may be or may comprise at least one photodetector,preferably inorganic photodetectors, more preferably inorganicsemiconductor photodetectors, most preferably silicon photodetectors.Specifically, the optical sensor 126 may be sensitive in the infraredspectral range. The optical sensor 126 may comprise at least one sensorelement comprising a matrix of pixels. All pixels of the matrix or atleast a group of the optical sensors of the matrix specifically may beidentical. Groups of identical pixels of the matrix specifically may beprovided for different spectral ranges, or all pixels may be identicalin terms of spectral sensitivity. Further, the pixels may be identicalin size and/or with regard to their electronic or optoelectronicproperties. Specifically, the optical sensor 126 may be or may compriseat least one array of inorganic photodiodes which are sensitive in theinfrared spectral range, preferably in the range of 700 nm to 3.0micrometers. Specifically, the optical sensor 126 may be sensitive inthe part of the near infrared region where silicon photodiodes areapplicable specifically in the range of 700 nm to 1100 nm. Infraredoptical sensors which may be used for optical sensors may becommercially available infrared optical sensors, such as infraredoptical sensors commercially available under the brand name Hertzstueck™from trinamiX™ GmbH, D-67056 Ludwigshafen am Rhein, Germany. Thus, as anexample, the optical sensor 126 may comprise at least one optical sensorof an intrinsic photovoltaic type, more preferably at least onesemiconductor photodiode selected from the group consisting of: a Gephotodiode, an InGaAs photodiode, an extended InGaAs photodiode, an InAsphotodiode, an InSb photodiode, a HgCdTe photodiode. Additionally oralternatively, the optical sensor may comprise at least one opticalsensor of an extrinsic photovoltaic type, more preferably at least onesemiconductor photodiode selected from the group consisting of: a Ge:Auphotodiode, a Ge:Hg photodiode, a Ge:Cu photodiode, a Ge:Zn photodiode,a Si:Ga photodiode, a Si:As photodiode. Additionally or alternatively,the optical sensor 126 may comprise at least one photoconductive sensorsuch as a PbS or PbSe sensor, a bolometer, preferably a bolometerselected from the group consisting of a VO bolometer and an amorphous Sibolometer.

Specifically, the optical sensor 126 may be sensitive in the nearinfrared region. Specifically, the optical sensor 126 may be sensitivein the part of the near infrared region where silicon photodiodes areapplicable specifically in the range of 700 nm to 1000 nm. The opticalsensor 126, specifically, may be sensitive in the infrared spectralrange, specifically in the range of 780 nm to 3.0 micrometers. Forexample, the optical sensor 126 may be or may comprise at least oneelement selected from the group consisting of a CCD sensor element, aCMOS sensor element, a photodiode, a photocell, a photoconductor, aphototransistor or any combination thereof. Any other type ofphotosensitive element may be used. The photosensitive element generallymay fully or partially be made of inorganic materials and/or may fullyor partially be made of organic materials. Most commonly, one or morephotodiodes may be used, such as commercially available photodiodes,e.g. inorganic semiconductor photodiodes.

The camera 112 further may comprise at least one transfer device (notshown here). The camera 112 may comprise at least one optical elementselected from the group consisting of: transfer device, such as at leastone lens and/or at least one lens system, at least one diffractiveoptical element. The transfer device may be adapted to guide the lightbeam onto the optical sensor 126. The transfer device specifically maycomprise one or more of: at least one lens, for example at least onelens selected from the group consisting of at least one focus-tunablelens, at least one aspheric lens, at least one spheric lens, at leastone Fresnel lens; at least one diffractive optical element; at least oneconcave mirror; at least one beam deflection element, preferably atleast one mirror; at least one beam splitting element, preferably atleast one of a beam splitting cube or a beam splitting mirror; at leastone multi-lens system. The transfer device may have a focal length.Thus, the focal length constitutes a measure of an ability of thetransfer device to converge an impinging light beam. Thus, the transferdevice may comprise one or more imaging elements which can have theeffect of a converging lens. By way of example, the transfer device canhave one or more lenses, in particular one or more refractive lenses,and/or one or more convex mirrors. In this example, the focal length maybe defined as a distance from the center of the thin refractive lens tothe principal focal points of the thin lens. For a converging thinrefractive lens, such as a convex or biconvex thin lens, the focallength may be considered as being positive and may provide the distanceat which a beam of collimated light impinging the thin lens as thetransfer device may be focused into a single spot. Additionally, thetransfer device can comprise at least one wavelength-selective element,for example at least one optical filter. Additionally, the transferdevice can be designed to impress a predefined beam profile on theelectromagnetic radiation, for example, at the location of the sensorregion and in particular the sensor area. The abovementioned optionalembodiments of the transfer device can, in principle, be realizedindividually or in any desired combination.

The transfer device may have an optical axis. The transfer device mayconstitute a coordinate system, wherein a longitudinal coordinate is acoordinate along the optical axis and wherein d is a spatial offset fromthe optical axis. The coordinate system may be a polar coordinate systemin which the optical axis of the transfer device forms a z-axis and inwhich a distance from the z-axis and a polar angle may be used asadditional coordinates. A direction parallel or antiparallel to thez-axis may be considered a longitudinal direction, and a coordinatealong the z-axis may be considered a longitudinal coordinate. Anydirection perpendicular to the z-axis may be considered a transversaldirection, and the polar coordinate and/or the polar angle may beconsidered a transversal coordinate.

The camera 112 is configured for determining at least one image of thescene, in particular the first image. The scene may refer to a spatialregion. The scene may comprise the face under authentication and asurrounding environment. The first image itself may comprise pixels, thepixels of the image correlating to pixels of the matrix of the sensorelement. The first image is at least one two-dimensional image havinginformation about transversal coordinates such as the dimensions ofheight and width.

The face detection step 110 comprises detecting the face in the firstimage by identifying in the first image the at least one pre-defined orpre-determined geometrical feature characteristic for faces by using theat least one processing unit 114. As an example, the at least oneprocessing unit 114 may comprise a software code stored thereoncomprising a number of computer commands. The processing unit 114 mayprovide one or more hardware elements for performing one or more of thenamed operations and/or may provide one or more processors with softwarerunning thereon for performing one or more of the named operations.Operations, including evaluating the images may be performed by the atleast one processing unit 114. Thus, as an example, one or moreinstructions may be implemented in software and/or hardware. Thus, as anexample, the processing unit 114 may comprise one or more programmabledevices such as one or more computers, application-specific integratedcircuits (ASICs), Digital Signal Processors (DSPs), or FieldProgrammable Gate Arrays (FPGAs) which are configured to perform theabove-mentioned evaluation. Additionally or alternatively, however, theprocessing unit may also fully or partially be embodied by hardware. Theprocessing unit 114 and the camera 112 may fully or partially beintegrated into a single device. Thus, generally, the processing unit114 also may form part of the camera 112. Alternatively, the processingunit 114 and the camera 112 may fully or partially be embodied asseparate devices.

The detecting of the face in the first image may comprise identifyingthe at least one predefined or pre-determined geometrical featurecharacteristic for faces. The geometrical feature characteristic forfaces may be at least one geometry-based feature which describe theshape of the face and its components, in particular one or more of nose,eyes, mouth or eyebrow and the like. The processing unit 114 maycomprise at least one database wherein the geometrical featurecharacteristic for faces are stored such as in a lookup table.Techniques for identifying the at least one pre-defined orpre-determined geometrical feature characteristic for faces aregenerally known to the skilled person. For example, the face detectionmay be performed as described in Masi, Lacopo, et al. “Deep facerecognition: A survey” 2018 31st SIBGRAPI conference on graphics,patterns and images (SIBGRAPI), IEEE, 2018, the full content of which isincluded by reference.

The processing unit 114 may be configured for performing at least oneimage analysis and/or image processing in order to identify thegeometrical feature. The image analysis and/or image processing may useat least one feature detection algorithm. The image analysis and/orimage processing may comprise one or more of the following: a filtering;a selection of at least one region of interest; a background correction;a decomposition into color channels; a decomposition into hue,saturation, and/or brightness channels; a frequency decomposition; asingular value decomposition; applying a blob detector; applying acorner detector; applying a Determinant of Hessian filter; applying aprinciple curvaturebased region detector; applying a gradient locationand orientation histogram algorithm; applying a histogram of orientedgradients descriptor; applying an edge detector; applying a differentialedge detector; applying a Canny edge detector; applying a Laplacian ofGaussian filter; applying a Difference of Gaussian filter; applying aSobel operator; applying a Laplace operator; applying a Scharr operator;applying a Prewitt operator; applying a Roberts operator; applying aKirsch operator; applying a high-pass filter; applying a low-passfilter; applying a Fourier transformation; applying aRadon-transformation; applying a Hough-transformation; applying awavelet-transformation; a thresholding; creating a binary image. Theregion of interest may be determined manually by a user or may bedetermined automatically, such as by recognizing a feature within thefirst image.

Specifically subsequent to the face detection step 110, the skindetection step 116 may be performed comprising projecting at least oneillumination pattern comprising a plurality of illumination features onthe scene by using the at least one illumination unit 118. However,embodiments are feasible wherein the skin detection step 116 isperformed before the face detection step 110.

The illumination unit 118 may be configured for providing theillumination pattern for illumination of the scene. The illuminationunit 118 may be adapted to directly or indirectly illuminating thescene, wherein the illumination pattern is remitted, in particularreflected or scattered, by surfaces of the scene and, thereby, is atleast partially directed towards the camera. The illumination unit 118may be configured for illuminating the scene, for example, by directinga light beam towards the scene, which reflects the light beam. Theillumination unit 118 may be configured for generating an illuminatinglight beam for illuminating the scene.

The illumination unit 118 may comprise at least one light source. Theillumination unit 118 may comprise a plurality of light sources. Theillumination unit 118 may comprise an artificial illumination source, inparticular at least one laser source and/or at least one incandescentlamp and/or at least one semiconductor light source, for example, atleast one lightemitting diode, in particular an organic and/or inorganiclight-emitting diode. The illumination unit 118 may be configured forgenerating the at least one illumination pattern in the infrared region.The illumination features may have wavelengths in a near infrared (NIR)regime. The illumination features may have wavelengths of about 940 nm.At this wavelength Melanin absorption runs out so that dark and lightcomplecion reflect light almost identical. However, other wavelength inthe NIR region may be possible such as one or more of 805 nm, 830 nm,835 nm, 850 nm, 905 nm, or 980 nm. Moreover, using light in the nearinfrared region allows that light is not or only weakly detected byhuman eyes and is still detectable by silicon sensors, in particularstandard silicon sensors.

The illumination unit 118 may be or may comprise at least one multiplebeam light source. For example, the illumination unit 118 may compriseat least one laser source and one or more diffractive optical elements(DOEs). Specifically, the illumination unit 118 may comprise at leastone laser and/or laser source. Various types of lasers may be employed,such as semiconductor lasers, double heterostructure lasers, externalcavity lasers, separate confinement heterostructure lasers, quantumcascade lasers, distributed bragg reflector lasers, polariton lasers,hybrid silicon lasers, extended cavity diode lasers, quantum dot lasers,volume Bragg grating lasers, Indium Arsenide lasers, transistor lasers,diode pumped lasers, distributed feedback lasers, quantum well lasers,interband cascade lasers, Gallium Arsenide lasers, semiconductor ringlaser, extended cavity diode lasers, or vertical cavity surface-emittinglasers. Additionally or alternatively, non-laser light sources may beused, such as LEDs and/or light bulbs. The illumination unit 118 maycomprise one or more diffractive optical elements (DOEs) adapted togenerate the illumination pattern. For example, the illumination unit118 may be adapted to generate and/or to project a cloud of points, forexample the illumination unit 118 may comprise one or more of at leastone digital light processing projector, at least one LCoS projector, atleast one spatial light modulator; at least one diffractive opticalelement; at least one array of light emitting diodes; at least one arrayof laser light sources. On account of their generally defined beamprofiles and other properties of handleability, the use of at least onelaser source as the illumination unit 118 is particularly preferred. Theillumination unit 118 may be integrated into a housing of the camera 112or may be separated from the camera 112.

The illumination pattern comprises at least one illumination featureadapted to illuminate at least one part of the scene. The illuminationpattern may comprise a single illumination feature. The illuminationpattern may comprise a plurality of illumination features. Theillumination pattern may be selected from the group consisting of: atleast one point pattern; at least one line pattern; at least one stripepattern; at least one checkerboard pattern; at least one patterncomprising an arrangement of periodic or non periodic features. Theillumination pattern may comprise regular and/or constant and/orperiodic pattern such as a triangular pattern, a rectangular pattern, ahexagonal pattern or a pattern comprising further convex tilings. Theillumination pattern may exhibit the at least one illumination featureselected from the group consisting of: at least one point; at least oneline; at least two lines such as parallel or crossing lines; at leastone point and one line; at least one arrangement of periodic ornon-periodic feature; at least one arbitrary shaped featured. Theillumination pattern may comprise at least one pattern selected from thegroup consisting of: at least one point pattern, in particular apseudo-random point pattern; a random point pattern or a quasi randompattern; at least one Sobol pattern; at least one quasiperiodic pattern;at least one pattern comprising at least one pre-known feature at leastone regular pattern; at least one triangular pattern; at least onehexagonal pattern; at least one rectangular pattern at least one patterncomprising convex uniform tilings; at least one line pattern comprisingat least one line; at least one line pattern comprising at least twolines such as parallel or crossing lines. For example, the illuminationunit 118 may be adapted to generate and/or to project a cloud of points.The illumination unit 118 may comprise the at least one light projectoradapted to generate a cloud of points such that the illumination patternmay comprise a plurality of point pattern. The illumination pattern maycomprise a periodic grid of laser spots. The illumination unit 118 maycomprise at least one mask adapted to generate the illumination patternfrom at least one light beam generated by the illumination unit 118.

The skin detection step 116 comprises determining the at least onesecond image, also denoted as reflection image, using the camera 112.The method may comprise determining plurality of second images. Thereflection features of the plurality of second images may be used forskin detection in step b) and/or for 3D detection in step c). Thereflection feature may be a feature in an image plane generated by thescene in response to illumination, specifically with at least oneillumination feature. Each of the reflection features comprises at leastone beam profile, also denoted reflection beam profile. The beam profileof the reflection feature may generally refer to at least one intensitydistribution of the reflection feature, such as of a light spot on theoptical sensor, as a function of the pixel. The beam profile may beselected from the group consisting of a trapezoid beam profile; atriangle beam profile; a conical beam profile and a linear combinationof Gaussian beam profiles.

The evaluation of the second image may comprise identifying thereflection features of the second image. The processing unit 114 may beconfigured for performing at least one image analysis and/or imageprocessing in order to identify the reflection features. The imageanalysis and/or image processing may use at least one feature detectionalgorithm. The image analysis and/or image processing may comprise oneor more of the following: a filtering; a selection of at least oneregion of interest; a formation of a difference image between an imagecreated by the sensor signals and at least one offset; an inversion ofsensor signals by inverting an image created by the sensor signals; aformation of a difference image between an image created by the sensorsignals at different times; a background correction; a decompositioninto color channels; a decomposition into hue; saturation; andbrightness channels; a frequency decomposition; a singular valuedecomposition; applying a blob detector; applying a corner detector;applying a Determinant of Hessian filter; applying a principlecurvature-based region detector; applying a maximally stable extremalregions detector; applying a generalized Hough-transformation; applyinga ridge detector; applying an affine invariant feature detector;applying an affine-adapted interest point operator; applying a Harrisaffine region detector; applying a Hessian affine region detector;applying a scaleinvariant feature transform; applying a scale-spaceextrema detector; applying a local feature detector; applying speeded uprobust features algorithm; applying a gradient location and orientationhistogram algorithm; applying a histogram of oriented gradientsdescriptor; applying a Deriche edge detector; applying a differentialedge detector; applying a spatiotemporal interest point detector;applying a Moravec corner detector; applying a Canny edge detector;applying a Laplacian of Gaussian filter; applying a Difference ofGaussian filter; applying a Sobel operator; applying a Laplace operator;applying a Scharr operator; applying a Prewitt operator; applying aRoberts operator; applying a Kirsch operator; applying a high-passfilter; applying a low-pass filter; applying a Fourier transformation;applying a Radon-transformation; applying a Hough-transformation;applying a wavelet-transformation; a thresholding; creating a binaryimage. The region of interest may be determined manually by a user ormay be determined automatically, such as by recognizing a feature withinthe image generated by the optical sensor 126.

For example, the illumination unit 118 may be configured for generatingand/or projecting a cloud of points such that a plurality of illuminatedregions is generated on the optical sensor 126, for example the CMOSdetector. Additionally, disturbances may be present on the opticalsensor 126 such as disturbances due to speckles and/or extraneous lightand/or multiple reflections. The processing unit 114 may be adapted todetermine at least one region of interest, for example one or morepixels illuminated by the light beam which are used for determination ofthe longitudinal coordinate for the respective reflection feature, whichwill be described in more detail below. For example, the processing unit114 may be adapted to perform a filtering method, for example, ablob-analysis and/or an edge filter and/or object recognition method.

The processing unit 114 may be configured for performing at least oneimage correction. The image correction may comprise at least onebackground subtraction. The processing unit 114 may be adapted to removeinfluences from background light from the beam profile, for example, byan imaging without further illumination.

The processing unit 114 may be configured for determining the beamprofile of the respective reflection feature. The determining the beamprofile may comprise identifying at least one reflection featureprovided by the optical sensor 126 and/or selecting at least onereflection feature provided by the optical sensor 126 and evaluating atleast one intensity distribution of the reflection feature. As anexample, a region of the matrix may be used and evaluated fordetermining the intensity distribution, such as a three-dimensionalintensity distribution or a two-dimensional intensity distribution, suchas along an axis or line through the matrix. As an example, a center ofillumination by the light beam may be determined, such as by determiningthe at least one pixel having the highest illumination, and acrosssectional axis may be chosen through the center of illumination.The intensity distribution may an intensity distribution as a functionof a coordinate along this cross-sectional axis through the center ofillumination. Other evaluation algorithms are feasible.

The processing unit 114 is configured for determining a first beamprofile information of at least one of the reflection features locatedinside an image region of the second image corresponding to an imageregion of the first image comprising the identified geometrical featureby analysis of its beam profile. The method may comprise identifying theimage region of the second image corresponding to the image region ofthe first image comprising the identified geometrical feature.Specifically, the method may comprise matching pixels of the first imageand the second image and selecting the pixels of the second imagecorresponding to the image region of the first image comprising theidentified geometrical feature. The method may comprise considering inaddition further reflection features located outside said image regionof the second image.

The beam profile information may be or may comprise arbitraryinformation and/or property derived from and/or relating to the beamprofile of the reflection feature. The first and the second beam profileinformation may be identical or may be different. For example, the firstbeam profile information may be an intensity distribution, a reflectionprofile, a center of intensity, a material feature. For skin detectionin step b) 116, beam profile analysis may be used. Specifically, beamprofile analysis makes use of reflection properties of coherent lightprojected onto object surfaces to classify materials. The classificationof materials may be performed as described in WO 2020/187719, in EPapplication 20159984.2 filed on Feb. 28, 2020 and/or EP application 20154 961.5 filed on Jan. 31, 2020, the full content of which is includedby reference. Specifically, a periodic grid of laser spots, e.g. ahexagonal grid as described in EP application 20 170 905.2 filed on Apr.22, 2020, is projected and the reflection image is recorded with thecamera. Analyzing the beam profile of each reflection feature recordedby the camera may be performed by feature-based methods. With respect tofeature-based methods reference is made to the description above. Thefeature based methods may be used in combination with machine learningmethods which may allow parametrization of a skin classification model.Alternatively or in combination, convolutional neuronal networks may beutilized to classify skin by using the reflection images as an input.

The skin detection step 116 may comprise determining at least onematerial property of the reflection feature from the beam profileinformation by using the processing unit 114. Specifically, theprocessing unit 114 is configured for identifying a reflection featureas to be generated by illuminating biological tissue, in particularhuman skin, in case its reflection beam profile fulfills at least onepredetermined or predefined criterion. The at least one predetermined orpredefined criterion may be at least one property and/or value suitableto distinguish biological tissue, in particular human skin, from othermaterials. The predetermined or predefined criterion may be or maycomprise at least one predetermined or predefined value and/or thresholdand/or threshold range referring to a material property. The reflectionfeature may be indicated as to be generated by biological tissue in casethe reflection beam profile fulfills the at least one predetermined orpredefined criterion. The processing unit is configured for identifyingthe reflection feature as to be non-skin otherwise. Specifically, theprocessing unit 114 may be configured for skin detection, in particularfor identifying if the detected face is human skin. The identificationif the material is biological tissue, in particular human skin, maycomprise to determining and/or validating whether a surface to beexamined or under test is or comprises biological tissue, in particularhuman skin, and/or to distinguish biological tissue, in particular humanskin, from other tissues, in particular other surfaces. The methodaccording to the present invention may allow for distinguishing humanskin from one or more of inorganic tissue, metal surfaces, plasticssurfaces, foam, paper, wood, a display, a screen, cloth. The methodaccording to the present invention may allow for distinguishing humanbiological tissue from surfaces of artificial or non-living objects.

The processing unit 114 may be configured for determining the materialproperty m of the surface remitting the reflection feature by evaluatingthe beam profile of the reflection feature. The material property may beat least one arbitrary property of the material configured forcharacterizing and/or identification and/or classification of thematerial. For example, the material property may be a property selectedfrom the group consisting of: roughness, penetration depth of light intothe material, a property characterizing the material as biological ornon-biological material, a reflectivity, a specular reflectivity, adiffuse reflectivity, a surface property, a measure for translucence, ascattering, specifically a back-scattering behavior or the like. The atleast one material property may be a property selected from the groupconsisting of: a scattering coefficient, a translucency, a transparency,a deviation from a Lambertian surface reflection, a speckle, and thelike. The determining at least one material property may compriseassigning the material property to the detected face. The processingunit 114 may comprise at least one database comprising a list and/ortable, such as a lookup list or a lookup table, of predefined and/orpredetermined material properties. The list and/or table of materialproperties may be determined and/or generated by performing at least onetest measurement, for example by performing material tests using sampleshaving known material properties. The list and/or table of materialproperties may be determined and/or generated at the manufacturer siteand/or by a user. The material property may additionally be assigned toa material classifier such as one or more of a material name, a materialgroup such as biological or non-biological material, translucent ornon-translucent materials, metal or non-metal, skin or non-skin, fur ornon-fur, carpet or non-carpet, reflective or non-reflective, specularreflective or non-specular reflective, foam or non-foam, hair ornon-hair, roughness groups or the like. The processing unit 114 maycomprise at least one database comprising a list and/or table comprisingthe material properties and associated material name and/or materialgroup.

While feature based approaches are accurate enough to differentiatebetween skin and surface-only scattering materials, the differentiationbetween skin and carefully selected attack materials, which involvevolume scattering as well, is more challenging. Step b) 116 may compriseusing artificial intelligence, in particular convolutional neuronalnetworks. Using reflection images as input for convolutional neuronalnetworks may enable the generation of classification models withsufficient accuracy to differentiate between skin and othervolume-scattering materials. Since only physically valid information ispassed to the network by selecting important regions in the reflectionimage, only compact training data sets may be needed. Additionally, verycompact network architectures can be generated.

Specifically, in the skin detection step 116 at least one parametrizedskin classification model may be used. The parametrized skinclassification model may be configured for classifying skin and othermaterials by using the second image as an input. The skin classificationmodel may be parametrized by using one or more of machine learning, deeplearning, neural networks, or other form of artificial intelligence. Themachine-learning may comprise a method of using artificial intelligence(AI) for automatically model building, in particular for parametrizingmodels. The skin classification model may comprise a classificationmodel configured for discriminating human skin from other materials. Theproperty characteristic for skin may be determined by applying anoptimization algorithm in terms of at least one optimization target onthe skin classification model. The machine learning may be based on atleast one neuronal network, in particular a convolutional neuralnetwork. Weights and/or topology of the neuronal network may bepre-determined and/or pre-defined. Specifically, the training of theskin classification model may be performed using machine-learning. Theskin classification model may comprise at least one machine-learningarchitecture and model parameters. For example, the machine-learningarchitecture may be or may comprise one or more of: linear regression,logistic regression, random forest, naive Bayes classifications, nearestneighbors, neural networks, convolutional neural networks, generativeadversarial networks, support vector machines, or gradient boostingalgorithms or the like. The training may comprise a process of buildingthe skin classification model, in particular determining and/or updatingparameters of the skin classification model. The skin classificationmodel may be at least partially data-driven. For example, the skinclassification model may be based on experimental data, such as datadetermined by illuminating a plurality of humans and artificial objectssuch as masks and recording the reflection pattern. For example, thetraining may comprise using at least one training dataset, wherein thetraining data set comprises images, in particular second images, of aplurality of humans and artificial objects with known material property.

The skin detection step 116 may comprises using at least one 2D face andfacial landmark detection algorithm configured for providing at leasttwo locations of characteristic points of a human face. For example, thelocations may be eye locations, forehead or cheeks. 2D face and faciallandmark detection algorithms may provide locations of characteristicpoints of a human face such as eye locations. Since there are subtledifferences in the reflection of the different zones of a face (forexample forehead or cheeks), region specific models can be trained. Inthe skin detection step 116, preferably at least one region specificparametrized skin classification model is used. The skin classificationmodel may comprise a plurality of region specific parametrized skinclassification models, such as for different regions, and/or the skinclassification model may be trained using region specific data such asby filtering the images used for training. For example, for training twodifferent regions may be used such as eye-cheek region to below the noseand, in particular in case not enough reflection features can beidentified within this region, the region of the forehead may be used.However, other regions may be possible, too.

The detected face is characterized as skin if the material propertycorresponds to at least one property characteristic for skin. Theprocessing unit 114 may be configured for identifying a reflectionfeature as to be generated by illuminating biological tissue, inparticular skin, in case its corresponding material property fulfillsthe at least one predetermined or predefined criterion. The reflectionfeature may be identified as to be generated by human skin in case thematerial property indicates “human skin”. The reflection feature may beidentified as to be generated by human skin in case the materialproperty is within at least one threshold and/or at least one range. Theat least one threshold value and/or range may be stored in a table or alookup table and may be determined e.g. empirically and may, as anexample, be stored in at least one data storage device of the processingunit. The processing unit 114 is configured for identifying thereflection feature as to be background otherwise. Thus, the processingunit 114 may be configured for assigning each projected spot with amaterial property, e.g. skin yes or no.

The 3D detection step 120 may be performed after the skin detection step116 and/or the face detection step 110. However, other embodiments arefeasible, in which the 3D detection step 120 is performed before theskin detection step 116 and/or the face detection step 110.

The 3D detection step 120 comprises determining the second beam profileinformation of at least four of the reflection features located insidethe image region of the second image corresponding to the image regionof the first image comprising the identified geometrical feature byanalysis of their beam profiles. The second beam profile information maycomprise a quotient Q of areas of the beam profile.

The analysis of the beam profile may comprise evaluating of the beamprofile and may comprise at least one mathematical operation and/or atleast one comparison and/or at least symmetrizing and/or at least onefiltering and/or at least one normalizing. For example, the analysis ofthe beam profile may comprise at least one of a histogram analysis step,a calculation of a difference measure, application of a neural network,application of a machine learning algorithm. The processing unit 114 maybe configured for symmetrizing and/or for normalizing and/or forfiltering the beam profile, in particular to remove noise or asymmetriesfrom recording under larger angles, recording edges or the like. Theprocessing unit 114 may filter the beam profile by removing high spatialfrequencies such as by spatial frequency analysis and/or medianfiltering or the like. Summarization may be performed by center ofintensity of the light spot and averaging all intensities at the samedistance to the center. The processing unit 114 may be configured fornormalizing the beam profile to a maximum intensity, in particular toaccount for intensity differences due to the recorded distance. Theprocessing unit 114 may be configured for removing influences frombackground light from the beam profile, for example, by an imagingwithout illumination.

The processing unit 114 may be configured for determining at least onelongitudinal coordinate z_(DPR) for reflection features located insidethe image region of the second image corresponding to the image regionof the first image comprising the identified geometrical feature byanalysis of the beam profile of the respective reflection feature. Theprocessing unit 114 may be configured for determining the longitudinalcoordinate z_(DPR) for the reflection features by using the so calleddepth-from-photon-ratio technique, also denoted as beam profileanalysis. With respect to depth-from-photon-ratio (DPR) techniquereference is made to WO 2018/091649 A1, WO 2018/091638 A1 and WO2018/091640 A1, the full content of which is included by reference.

The longitudinal coordinate for the reflection feature may be a distancebetween the optical sensor 126 and the point of the scene remitting thecorresponding illumination features. The analysis of the beam profile ofone of the reflection features may comprise determining at least onefirst area and at least one second area of the beam profile. The firstarea of the beam profile may be an area A1 and the second area of thebeam profile may be an area A2. The processing unit 114 may beconfigured for integrating the first area and the second area. Theprocessing unit 114 may be configured to derive a combined signal, inparticular a quotient Q, by one or more of dividing the integrated firstarea and the integrated second area, dividing multiples of theintegrated first area and the integrated second area, dividing linearcombinations of the integrated first area and the integrated secondarea. The processing unit 114 may configured for determining at leasttwo areas of the beam profile and/or to segment the beam profile in atleast two segments comprising different areas of the beam profile,wherein overlapping of the areas may be possible as long as the areasare not congruent. For example, the processing unit 114 may beconfigured for determining a plurality of areas such as two, three,four, five, or up to ten areas. The processing unit 114 may beconfigured for segmenting the light spot into at least two areas of thebeam profile and/or to segment the beam profile in at least two segmentscomprising different areas of the beam profile. The processing unit 114may be configured for determining for at least two of the areas anintegral of the beam profile over the respective area. The processingunit may be configured for comparing at least two of the determinedintegrals. Specifically, the processing unit 114 may be configured fordetermining at least one first area and at least one second area of thebeam profile. The area of the beam profile may be an arbitrary region ofthe beam profile at the position of the optical sensor used fordetermining the quotient Q. The first area of the beam profile and thesecond area of the beam profile may be one or both of adjacent oroverlapping regions. The first area of the beam profile and the secondarea of the beam profile may be not congruent in area. For example, theprocessing unit 114 may be configured for dividing a sensor region ofthe CMOS sensor into at least two sub-regions, wherein the processingunit may be configured for dividing the sensor region of the CMOS sensorinto at least one left part and at least one right part and/or at leastone upper part and at least one lower part and/or at least one inner andat least one outer part. Additionally or alternatively, the camera 112may comprise at least two optical sensors 126, wherein thelight-sensitive areas of a first optical sensor 126 and of a secondoptical sensor 126 may be arranged such that the first optical sensor126 is adapted to determine the first area of the beam profile of thereflection feature and that the second optical sensor 126 is adapted todetermine the second area of the beam profile of the reflection feature.The processing unit 114 may be adapted to integrate the first area andthe second area. The processing unit 114 may be configured for using atleast one predetermined relationship between the quotient Q and thelongitudinal coordinate for determining the longitudinal coordinate. Thepredetermined relationship may be one or more of an empiricrelationship, a semi-empiric relationship and an analytically derivedrelationship. The processing unit 114 may comprise at least one datastorage device for storing the predetermined relationship, such as alookup list or a lookup table.

The 3D detection step may comprise determining the at least one depthlevel from the second beam profile information of said reflectionfeatures by using the processing unit.

The processing unit 114 may be configured for determining the depth mapof at least parts of the scene by determining at least one depthinformation of the reflection features located inside the image regionof the second image corresponding to the image region of the first imagecomprising the identified geometrical feature. The processing unit 114may be configured for determining the depth information of thereflection features by one or more of the following techniques:depth-from-photon-ratio, structured light, beam profile analysis,time-of-flight, shape-from-motion, depth-from-focus, triangulation,depth-from-defocus, stereo sensors. The depth map may be a thinly filleddepth map comprising a few entries. Alternatively, the depth may becrowded comprising a large amount of entries.

The detected face is characterized as 3D object if the depth leveldeviates from a pre-determined or pre-defined depth level of planeobjects. Step c) 120 may comprise using 3D topology data of the face infront of the camera. The method may comprise determining a curvaturefrom the at least four of the reflection features located inside theimage region of the second image corresponding to the image region ofthe first image comprising the identified geometrical feature. Themethod may comprise comparing the curvature determined from the at leastfour of the reflection features to the pre-determined or pre-defineddepth level of plane objects. If the curvature exceeds an assumedcurvature of plane object the detected face may be characterized as 3Dobject, otherwise as plane object. The pre-determined or pre-defineddepth level of plane objects may be stored in at least one data storageof the processing unit such as a lookup list or a lookup table. Thepre-determined or predefined level of plane objects may beexperimentally determined and/or may be a theoretical level of planeobjects. The pre-determined or pre-defined depth level of plane objectsmay be at least one limit for at least one curvature and/or a range forat least one curvature.

3D features determined step c) 120 may allow differentiating betweenhigh quality photographs and a 3D face-like structure. The combinationof steps b) 116 and c) 120 may allow strengthening reliability of theauthentication with respect to attacks. 3D features can be combined withmaterial features to increase the security level. Since the samecomputational pipeline can be used to generate the input data for theskin classification and the generation of the 3D point cloud, bothproperties can be calculated from the same frame with low computationaleffort.

Preferably subsequent to steps a) 110, b) 116 and c) 120 theauthentication step 122 may be performed. The authentication step 122may be performed partially after each of steps a) to c). Theauthentication may be aborted in case in step a) 110 no face is detectedand/or in step b) 116 the reflection features is determined not to begenerated by skin and/or in step c) 120 the depth map refers to a planeobject. The authentication step comprises authenticating the detectedface by using at least one authentication unit if in step b) 116 thedetected face is characterized as skin and in step c) 122 the detectedface is characterized as 3D object.

Steps a) to d) may be performed by using at least one device, forexample the at least one mobile device 124 such as a mobile phone,smartphone and the like, wherein access of the device is secured byusing face authentication. Other devices may be possible, too such as anaccess control device controlling access to buildings, machines,automobiles and the like. The method may comprise permitting access tothe device if the detected face is authenticated.

The method may comprise at least one enrollment step. In the enrollmentstep a user of the device may be enrolled. The enrolling may comprise aprocess of registering and/or signing up and/or teach in of a user forsubsequent usage of the device. Usually, enrolling may be performed atfirst use of the device and/or for initiating the device. However,embodiments are feasible in which a plurality of users may be enrolled,e.g. successively, such that the enrolling may be performed and/orrepeated at an arbitrary time during usage of the device. The enrollingmay comprise generating a user account and/or user profile. Theenrolling may comprise entering and storing user data, in particularimage data, via at least one user interface. Specifically, at least one2D image of the user is stored in at least one database. The enrollmentstep may comprise imaging at least one image of the user, in particulara plurality of images. The images may be recorded from differentdirection and/or the user may change his orientation. Additionally, theenrollment step may comprise generating at least one 3D image and/or adepth map for the user which may be used in step d) for comparison. Thedatabase may a database of the device, e.g. of the processing unit 114,and/or may be an external database such as a cloud. The method comprisesidentifying the user by comparing the 2D image of the user with thefirst image. The method according to the present invention may allowsignificantly improving the presentation attack detection capabilitiesof biometric authentication methods. In order improve the overallauthentication, person specific material fingerprints as well as 3Dtopological features may be stored during the enrollment process inaddition to the 2D image of the user. This may allow a multifactorauthentication within one device by using 2D, 3D and material-derivedfeatures.

The method according to the present invention using beam profileanalysis technology may provide a concept to reliably detect human skinby analyzing reflections of laser spots, in particular in the NIRregime, on a face and distinguish it from reflections coming from attackmaterials that were produced to mimic a face. Additionally, beam profileanalysis simultaneously provides depth information by analyzing the samecamera frame. Therefore, 3D as well as skin security features may beprovided by the exact same technology.

Since also the 2D image of the face can be recorded by simply switchingoff the laser illumination, a fully secure face recognition pipeline canbe established solving the above-stated problem.

The reflection properties of human skin with respect to ethnic originbecome more similar when the laser wavelength is shifted towards the NIRregime. At a wavelength of 940 nm differences are at a minimum.Accordingly, different ethnic origins do not play a role for skinauthentication.

No time-consuming analysis of a series of frames may be necessary sincepresentation attack detection (via skin classification) is provided byjust one frame. A time frame for performing the complete method may be≤500 ms, preferably ≤250 ms. However, embodiments may be feasible inwhich the skin detection may be performed using a plurality of frames.Depending on confidence for identifying reflection features in thesecond image, and speed of the method, the method may comprise samplingreflection features over several frames in order to reach a more stableclassification.

FIG. 3 shows experimental results, in particular density as a functionof skin score. On the x-axis the score is shown and on the y-axis thefrequency. The score is a measure for the classification quality and hasa value range between 0 and 1, wherein 1 indicates very highskin-similarity and 0 very low skin-similarity. A threshold for decisionmay be around 0.5. A reference distribution of skin-scores for bona fidepresentations has been generated using 10 subjects. Skin-scores havealso been recorded for presentation attacks (PA) of Level A, Level B andLevel C (as defined in the relevant ISO standards). The experimentalsetup (target of evaluation, ToE) included a proprietary hardwaredevice, e.g. as described in FIG. 2 , which includes the necessarysensors as well as the computational platform on which the PAD softwareis executed. The ToE was tested using six species of target of Level APAls (presentation attack instruments), five PAI species of Level B, andone PAI species of Level C. For each PAI species, 10 PAls have beenused. The PAI species used in this study are listed in the table below.In the table, APCER is the Attack Presentation Classification ErrorRate, which refers to the number of successful attacks/all attacks*100.In the table, BPCER is the Bona Fide Presentation Classification ErrorRate, which refers to the number of rejected unlock-attempts/allunlock-attempts*100 Level A and Level B attacks are all based on 2DPAls, whereas for Level C attacks, based on 3D masks. For Level Cattacks, custom rigid masks constructed using a 3D printer were used. Atest crew of 10 subjects has been used to obtain the referencedistribution of skin-scores for bona fide presentations.

#PAI PAI Level species species #objects #presentations Results A 6 4printed, 60 300 (60 × 5) APCER = 0% 2 digital replay (normal quality) B5 4 printed, 50 250 (50 × 5) APCER = 0% 1 digital replay (optimized forspoof) C 1 3D printed 10 50 (10 × 5) APCER = 0% mask Bona — — 10 50 (10× 5) BPCER = 0% Fide

The experiments with these PAls show that the two classes ofpresentations (bona fide or PA) are clearly distinguishable based on theskin-score. Clear distinction between paper, 3D print and skin ispossible using the method according to the present invention.

LIST OF REFERENCE NUMBERS

-   -   110 face detection step    -   112 camera    -   114 processing unit    -   116 skin detection step    -   118 illumination unit    -   120 3D detection step    -   122 authentication step    -   124 mobile device    -   126 optical sensor

1. A method for face authentication comprising the following steps: a. at least one face detection step comprising determining at least one first image by using at least one camera; b. determining at least one material property from a second image, wherein the second image is recorded while projecting at least one illumination pattern comprising a plurality of illumination features on the scene, and c. at least one authentication step comprising authenticating the detected face by using the face detection from step a and the material property of step b.
 2. The method according to claim 1, wherein the authentication step comprises authentication of the detected face if the material property corresponds to at least one property characteristic for skin.
 3. The method according to claim 1, wherein the illumination comprises light with a wavelength of 700 nm to 1100 nm.
 4. The method according to claim 1, wherein the illumination comprises light with a wavelength of 940 nm.
 5. The method according to claim 1, wherein the face detection step comprises detecting the face in the first image by identifying in the first image at least one geometrical feature characteristic for a face.
 6. The method according to claim 1, wherein the face detection step comprises a selection of at least one region of interest.
 7. The method according to claim 1, wherein the illumination pattern is a periodic pattern.
 8. The method according to claim 6, wherein the illumination pattern is a triangular pattern, a rectangular pattern, or a hexagonal pattern.
 9. The method according to claim 6, wherein the illumination pattern comprises a periodic grid of laser spots.
 10. The method according to claim 1, wherein determining the material property is based on roughness, penetration depth of light into the material, a specular reflectivity, a diffuse reflectivity, a measure for translucence, or a back-scattering behavior.
 11. The method according to claim 1, wherein determining the material property involves a convolutional neuronal network.
 12. The method according to claim 1, wherein determining the material property comprises a material classification.
 13. The method according to claim 1, wherein the method further comprises a distance measurement.
 14. The method according to claim 1, wherein the method further comprises determining a depth map.
 15. The method according to claim 1, wherein the method is completed in less than or equal to 250 ms.
 16. A face authentication system comprising a. a camera for recording an image of a scene, b. an illumination unit for projecting at least one illumination pattern comprising a plurality of illumination features on the scene, and c. a processor for executing an authorization comprising i. determining a face from a first image recorded by the camera, ii. determining a material property from a second image, wherein the second image is recorded by the camera while the illumination unit projects light on the scene, and iii. authenticating the face by using the face determination from the first image and the material property from the second image.
 17. The face authentication system according to claim 16, wherein the face authentication system is integrated into a mobile device.
 18. The face authentication system according to claim 16, wherein the illumination unit comprises a diffractive optical element.
 19. The face authentication system according to claim 16, wherein the illumination unit comprises at least one array of light emitting diodes or at least one array of laser light sources.
 20. The face authentication system according to claim 16, wherein the illumination unit projects light with a wavelength of 940 nm. 